2024-04-25T00:54:13.812 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:54:13.812 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} --owner scheduled_teuthology@teuthology --verbose --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302 -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/orig.config.yaml 2024-04-25T00:54:13.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:54:13.941 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302 branch: squid description: fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672302' kclient-workload: sequential: - kclient: - client.1 - workunit: clients: client.1: - suites/dbench.sh kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 valgrind: exit_on_first_error: false ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCVQ59ja3FdqN+8CNqzAlGvQtPuTBLf2jfZ8JYDtY3/Jgcfe4zv3FAMmEKWv8Z523cELqFpb0Ps/7/q7N6BK9Kc= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqSqd/60S1Qait7GDVjA8IWPelcig9o1NLqgKeHwnpIDWko8RhGZ/RcwxpmpFE4LEGx+speFRGVhoPK2tmR8hY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - parallel: - user-workload - kclient-workload teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology user-workload: sequential: - ceph-fuse: - client.0 - workunit: clients: client.0: - suites/iozone.sh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:54:13.941 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:54:13.941 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:54:13.942 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:54:13.943 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:54:13.967 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:54:13.967 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:54:13.967 INFO:teuthology.packaging:ref: None 2024-04-25T00:54:13.967 INFO:teuthology.packaging:tag: None 2024-04-25T00:54:13.967 INFO:teuthology.packaging:branch: squid 2024-04-25T00:54:13.967 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:54:13.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T00:54:14.167 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T00:54:14.169 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:54:14.175 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:54:14.175 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:54:14.220 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:54:14.237 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:54:14.243 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:54:14.265 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:49:05.890403', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCVQ59ja3FdqN+8CNqzAlGvQtPuTBLf2jfZ8JYDtY3/Jgcfe4zv3FAMmEKWv8Z523cELqFpb0Ps/7/q7N6BK9Kc='} 2024-04-25T00:54:14.285 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:49:05.891657', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqSqd/60S1Qait7GDVjA8IWPelcig9o1NLqgKeHwnpIDWko8RhGZ/RcwxpmpFE4LEGx+speFRGVhoPK2tmR8hY='} 2024-04-25T00:54:14.285 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:54:14.291 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T00:54:14.291 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-04-25T00:54:14.291 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:54:14.359 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3013806040>, signals=[15]) 2024-04-25T00:54:14.359 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:54:14.365 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:54:14.365 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T00:54:14.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:54:14.442 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T00:54:14.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:54:14.521 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:54:14.529 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-04-25T00:54:14.547 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-04-25T00:54:14.547 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-04-25T00:54:14.603 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-04-25T00:54:14.603 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-04-25T00:54:14.603 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-04-25T00:54:14.603 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:54:14.604 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:54:14.605 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-04-25T00:54:14.621 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-04-25T00:54:14.639 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-04-25T00:54:14.639 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-04-25T00:54:14.696 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-04-25T00:54:14.696 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-04-25T00:54:14.696 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T00:54:14.697 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T00:54:14.697 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-04-25T00:54:14.721 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:54:14.731 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:54:14.736 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:54:14.736 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:54:14.739 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:54:14.755 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:54:14.761 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:54:14.761 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:54:14.796 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:54:14.832 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:54:14.883 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:54:14.897 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:54:14.897 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-04-25T00:54:14.898 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T00:54:14.898 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:54:14.898 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:54:14.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:14.915 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-04-25T00:54:14.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:14.916 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-04-25T00:54:14.972 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:54:14.972 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.14.0-437.el9.x86_64 2024-04-25T00:54:14.972 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum install -y kernel 2024-04-25T00:54:14.974 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-437.el9.x86_64 2024-04-25T00:54:14.974 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-437.el9.x86_64 2024-04-25T00:54:14.974 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-04-25T00:54:17.260 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T00:54:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.0 MB 00:01 2024-04-25T00:54:19.236 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T00:54:20.340 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T00:54:25.215 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-25T00:54:26.849 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-25T00:54:27.037 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-04-25T00:54:28.880 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T00:54:28.954 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T00:54:30.366 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T00:54:34.084 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T00:54:35.455 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-25T00:54:35.973 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T00:54:35.974 INFO:teuthology.orchestra.run.smithi171.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T00:54:36.061 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-25T00:54:36.062 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-04-25T00:54:36.063 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-25T00:54:36.162 DEBUG:teuthology.orchestra.run.smithi171:> echo no | sudo yum reinstall kernel || true 2024-04-25T00:54:36.664 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:54:33 AM UTC. 2024-04-25T00:54:36.788 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-25T00:54:36.798 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-04-25T00:54:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-04-25T00:54:36.798 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-04-25T00:54:36.798 INFO:teuthology.orchestra.run.smithi171.stdout:Reinstalling: 2024-04-25T00:54:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:54:36.798 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:54:36.799 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-25T00:54:36.799 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-04-25T00:54:36.799 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:54:36.799 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 255 k 2024-04-25T00:54:36.799 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 0 2024-04-25T00:54:36.799 INFO:teuthology.orchestra.run.smithi171.stderr:Operation aborted. 2024-04-25T00:54:36.865 INFO:teuthology.orchestra.run.smithi171.stdout:Is this ok [y/N]: 2024-04-25T00:54:36.866 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum reinstall -y kernel || true 2024-04-25T00:54:37.303 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T00:54:37.303 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T00:54:37.357 INFO:teuthology.orchestra.run.smithi171.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 12:54:33 AM UTC. 2024-04-25T00:54:37.395 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-25T00:54:37.396 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-04-25T00:54:37.396 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-25T00:54:37.473 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout: Package Architecture Version Repository Size 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:Reinstalling: 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:================================================================================ 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 255 k 2024-04-25T00:54:37.474 INFO:teuthology.orchestra.run.smithi171.stdout:Installed size: 0 2024-04-25T00:54:37.475 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-25T00:54:37.507 DEBUG:teuthology.orchestra.run.smithi129:> echo no | sudo yum reinstall kernel || true 2024-04-25T00:54:38.044 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:54:35 AM UTC. 2024-04-25T00:54:38.086 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-437.el9.x86_64.rpm 499 kB/s | 255 kB 00:00 2024-04-25T00:54:38.086 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:54:38.086 INFO:teuthology.orchestra.run.smithi171.stdout:Total 416 kB/s | 255 kB 00:00 2024-04-25T00:54:38.100 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-25T00:54:38.162 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-25T00:54:38.204 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalling: 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 255 k 2024-04-25T00:54:38.205 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2024-04-25T00:54:38.206 INFO:teuthology.orchestra.run.smithi129.stderr:Operation aborted. 2024-04-25T00:54:38.268 INFO:teuthology.orchestra.run.smithi129.stdout:Is this ok [y/N]: 2024-04-25T00:54:38.269 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum reinstall -y kernel || true 2024-04-25T00:54:38.301 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-25T00:54:38.301 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-25T00:54:38.306 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-25T00:54:38.306 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-25T00:54:38.435 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-25T00:54:38.498 INFO:teuthology.orchestra.run.smithi171.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:54:38.498 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:54:38.762 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 12:54:35 AM UTC. 2024-04-25T00:54:38.880 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-25T00:54:38.880 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-25T00:54:38.880 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalling: 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 255 k 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2024-04-25T00:54:38.881 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-04-25T00:54:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:54:39.116 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:54:39.649 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-437.el9.x86_64.rpm 408 kB/s | 255 kB 00:00 2024-04-25T00:54:39.649 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:54:39.649 INFO:teuthology.orchestra.run.smithi129.stdout:Total 332 kB/s | 255 kB 00:00 2024-04-25T00:54:39.665 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-04-25T00:54:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:54:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:54:39.799 INFO:teuthology.orchestra.run.smithi171.stdout:Reinstalled: 2024-04-25T00:54:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T00:54:39.799 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:54:39.799 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-25T00:54:39.862 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-04-25T00:54:39.862 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-04-25T00:54:39.866 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-04-25T00:54:39.867 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-04-25T00:54:39.990 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T00:54:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-04-25T00:54:40.036 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T00:54:40.037 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T00:54:40.037 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T00:54:40.037 DEBUG:teuthology.parallel:result is None 2024-04-25T00:54:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:54:40.066 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:54:40.720 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:54:40.720 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T00:54:41.455 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T00:54:41.456 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:54:41.456 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalled: 2024-04-25T00:54:41.456 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T00:54:41.456 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:54:41.456 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-25T00:54:41.644 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T00:54:41.731 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T00:54:41.731 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T00:54:41.731 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T00:54:41.731 DEBUG:teuthology.parallel:result is None 2024-04-25T00:54:41.731 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:54:41.739 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:54:41.739 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:54:41.742 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:54:41.761 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:54:41.808 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:54:41.815 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:54:41.815 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:54:41.817 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:54:41.854 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:54:41.863 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:54:41.863 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:54:41.884 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:54:41.915 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:54:41.923 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:54:41.930 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:54:41.939 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:54:41.941 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:54:41.949 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:54:41.949 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:54:41.974 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:54:42.007 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:54:42.017 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:54:42.017 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:54:42.038 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:54:42.065 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:54:42.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:54:42.190 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:54:42.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:54:42.312 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:54:42.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:54:42.375 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:54:42.419 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:54:42.490 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:54:42.535 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:54:42.576 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T00:54:42.577 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:54:42.642 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-04-25T00:54:42.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-04-25T00:54:42.677 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:54:42.719 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:54:42.956 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:54:42.964 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:54:42.965 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:54:42.976 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:54:42.985 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T00:54:42.986 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-04-25T00:54:43.053 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-04-25T00:54:43.055 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-04-25T00:54:43.105 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-04-25T00:54:43.107 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:54:43.107 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-04-25T00:54:43.128 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-04-25T00:54:43.128 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-04-25T00:54:43.151 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-04-25T00:54:43.152 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-04-25T00:54:43.152 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:54:43.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:43.193 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:54:43.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:43.217 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:54:43.217 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:54:43.257 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-04-25T00:54:43.282 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:54:43.301 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:54:43.320 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:54:43.321 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:58:11.360 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-04-25T00:58:11.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:58:11.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:58:11.437 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-04-25T00:58:11.513 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-25T00:58:11.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-25T00:58:11.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:58:11.592 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-04-25T00:58:11.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-25T00:58:11.670 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:58:11.681 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:58:11.681 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:58:11.681 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:58:11.683 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:58:11.684 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:58:11.708 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:58:11.721 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:58:11.738 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:58:11.744 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-04-25T00:58:11.752 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:58:11.754 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-04-25T00:58:11.767 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:58:11.775 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-04-25T00:58:11.780 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:58:11.785 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-04-25T00:58:11.797 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:58:11.810 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:58:11.840 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:58:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:58:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-25T00:58:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.844 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.865 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:58:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:58:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-25T00:58:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.870 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:58:11.870 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:58:11.881 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:58:11.881 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:58:11.881 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:58:11.881 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:58:11.887 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:58:11.887 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:58:11.887 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:58:11.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:58:11.890 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:58:11.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:58:12.034 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:58:12.035 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:58:12.109 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:58:12.109 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:58:12.235 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:58:12.235 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T00:58:12.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:58:12.261 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:58:12.261 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:58:12.322 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:58:12.323 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T00:58:12.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:58:12.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:58:12.347 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:58:12.348 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:58:12.401 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:58:12.411 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:58:12.464 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-04-25T00:58:12.465 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-04-25T00:58:12.486 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:58:12.550 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-04-25T00:58:12.551 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-04-25T00:58:12.690 INFO:teuthology.orchestra.run.smithi129.stdout:48 files removed 2024-04-25T00:58:12.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:58:12.779 INFO:teuthology.orchestra.run.smithi171.stdout:48 files removed 2024-04-25T00:58:12.806 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:58:13.584 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 137 kB/s | 83 kB 00:00 2024-04-25T00:58:13.816 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 111 kB/s | 83 kB 00:00 2024-04-25T00:58:14.242 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-04-25T00:58:14.358 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-04-25T00:58:14.635 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-25T00:58:14.809 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-25T00:58:15.226 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T00:58:15.625 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T00:58:19.553 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-25T00:58:23.896 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-25T00:58:25.697 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-25T00:58:29.414 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-04-25T00:58:32.960 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-04-25T00:58:34.688 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 5.4 kB/s | 1.7 kB 00:00 2024-04-25T00:58:36.826 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:58:36.826 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:58:36.834 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:58:36.835 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:58:36.836 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:58:36.837 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:58:36.838 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:58:36.841 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:58:36.841 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:58:36.901 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-25T00:58:36.910 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:58:36.911 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:58:36.912 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:58:36.913 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:58:36.914 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:58:36.915 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:58:36.916 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:58:36.918 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:58:36.919 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:58:36.920 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:58:36.921 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T00:58:36.922 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-04-25T00:58:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 4 Packages 2024-04-25T00:58:36.923 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:58:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 220 M 2024-04-25T00:58:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-25T00:58:37.699 INFO:teuthology.orchestra.run.smithi171.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-25T00:58:38.424 INFO:teuthology.orchestra.run.smithi171.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-25T00:58:38.508 INFO:teuthology.orchestra.run.smithi171.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.7 MB/s | 145 kB 00:00 2024-04-25T00:58:38.810 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-04-25T00:58:39.350 INFO:teuthology.orchestra.run.smithi171.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-25T00:58:39.500 INFO:teuthology.orchestra.run.smithi171.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-25T00:58:39.817 INFO:teuthology.orchestra.run.smithi171.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-25T00:58:40.434 INFO:teuthology.orchestra.run.smithi171.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.0 MB/s | 4.7 MB 00:00 2024-04-25T00:58:41.119 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T00:58:42.317 INFO:teuthology.orchestra.run.smithi171.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 4.3 MB/s | 21 MB 00:04 2024-04-25T00:58:42.434 INFO:teuthology.orchestra.run.smithi171.stdout:(9/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 5.1 MB/s | 10 MB 00:01 2024-04-25T00:58:42.459 INFO:teuthology.orchestra.run.smithi171.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 178 kB/s | 25 kB 00:00 2024-04-25T00:58:42.634 INFO:teuthology.orchestra.run.smithi171.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.2 MB/s | 222 kB 00:00 2024-04-25T00:58:42.726 INFO:teuthology.orchestra.run.smithi171.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 295 kB/s | 27 kB 00:00 2024-04-25T00:58:42.826 INFO:teuthology.orchestra.run.smithi171.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 318 kB/s | 32 kB 00:00 2024-04-25T00:58:43.068 INFO:teuthology.orchestra.run.smithi171.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.0 MB/s | 745 kB 00:00 2024-04-25T00:58:43.169 INFO:teuthology.orchestra.run.smithi171.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-25T00:58:43.286 INFO:teuthology.orchestra.run.smithi171.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.1 MB/s | 127 kB 00:00 2024-04-25T00:58:43.387 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-04-25T00:58:43.477 INFO:teuthology.orchestra.run.smithi171.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.6 MB/s | 501 kB 00:00 2024-04-25T00:58:44.261 INFO:teuthology.orchestra.run.smithi171.stdout:(18/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 3.8 MB/s | 17 MB 00:04 2024-04-25T00:58:44.370 INFO:teuthology.orchestra.run.smithi171.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 418 kB/s | 45 kB 00:00 2024-04-25T00:58:44.503 INFO:teuthology.orchestra.run.smithi171.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 997 kB/s | 133 kB 00:00 2024-04-25T00:58:44.612 INFO:teuthology.orchestra.run.smithi171.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.5 MB/s | 162 kB 00:00 2024-04-25T00:58:44.754 INFO:teuthology.orchestra.run.smithi171.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 4.2 MB/s | 5.3 MB 00:01 2024-04-25T00:58:44.779 INFO:teuthology.orchestra.run.smithi171.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.9 MB/s | 321 kB 00:00 2024-04-25T00:58:44.863 INFO:teuthology.orchestra.run.smithi171.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.7 MB/s | 299 kB 00:00 2024-04-25T00:58:44.888 INFO:teuthology.orchestra.run.smithi171.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 918 kB/s | 99 kB 00:00 2024-04-25T00:58:44.972 INFO:teuthology.orchestra.run.smithi171.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 787 kB/s | 85 kB 00:00 2024-04-25T00:58:45.114 INFO:teuthology.orchestra.run.smithi171.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.2 MB/s | 171 kB 00:00 2024-04-25T00:58:45.222 INFO:teuthology.orchestra.run.smithi171.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 294 kB/s | 32 kB 00:00 2024-04-25T00:58:45.322 INFO:teuthology.orchestra.run.smithi171.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.4 MB/s | 141 kB 00:00 2024-04-25T00:58:45.639 INFO:teuthology.orchestra.run.smithi171.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.1 MB/s | 3.1 MB 00:00 2024-04-25T00:58:46.181 INFO:teuthology.orchestra.run.smithi171.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 4.1 MB/s | 3.5 MB 00:00 2024-04-25T00:58:46.290 INFO:teuthology.orchestra.run.smithi171.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.2 MB/s | 249 kB 00:00 2024-04-25T00:58:46.366 INFO:teuthology.orchestra.run.smithi171.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 658 kB/s | 49 kB 00:00 2024-04-25T00:58:46.424 INFO:teuthology.orchestra.run.smithi171.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 264 kB/s | 15 kB 00:00 2024-04-25T00:58:46.491 INFO:teuthology.orchestra.run.smithi171.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 264 kB 00:00 2024-04-25T00:58:46.675 INFO:teuthology.orchestra.run.smithi171.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-25T00:58:46.800 INFO:teuthology.orchestra.run.smithi171.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-04-25T00:58:46.826 INFO:teuthology.orchestra.run.smithi171.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 121 kB/s | 18 kB 00:00 2024-04-25T00:58:47.168 INFO:teuthology.orchestra.run.smithi171.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 219 kB/s | 80 kB 00:00 2024-04-25T00:58:47.359 INFO:teuthology.orchestra.run.smithi171.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-25T00:58:47.543 INFO:teuthology.orchestra.run.smithi171.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-25T00:58:47.585 INFO:teuthology.orchestra.run.smithi171.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-25T00:58:47.685 INFO:teuthology.orchestra.run.smithi171.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-25T00:58:47.794 INFO:teuthology.orchestra.run.smithi171.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-25T00:58:47.836 INFO:teuthology.orchestra.run.smithi171.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-04-25T00:58:47.903 INFO:teuthology.orchestra.run.smithi171.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-25T00:58:47.945 INFO:teuthology.orchestra.run.smithi171.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-25T00:58:48.070 INFO:teuthology.orchestra.run.smithi171.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-04-25T00:58:48.104 INFO:teuthology.orchestra.run.smithi171.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-25T00:58:48.379 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T00:58:48.613 INFO:teuthology.orchestra.run.smithi171.stdout:(50/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.8 MB/s | 48 MB 00:06 2024-04-25T00:58:48.637 INFO:teuthology.orchestra.run.smithi171.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 40 kB/s | 21 kB 00:00 2024-04-25T00:58:48.663 INFO:teuthology.orchestra.run.smithi171.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 180 kB/s | 107 kB 00:00 2024-04-25T00:58:48.738 INFO:teuthology.orchestra.run.smithi171.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-04-25T00:58:48.847 INFO:teuthology.orchestra.run.smithi171.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-25T00:58:48.947 INFO:teuthology.orchestra.run.smithi171.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-25T00:58:49.056 INFO:teuthology.orchestra.run.smithi171.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T00:58:49.156 INFO:teuthology.orchestra.run.smithi171.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2024-04-25T00:58:49.214 INFO:teuthology.orchestra.run.smithi171.stdout:(58/152): flex-2.6.4-9.el9.x86_64.rpm 521 kB/s | 313 kB 00:00 2024-04-25T00:58:49.331 INFO:teuthology.orchestra.run.smithi171.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.5 MB/s | 3.0 MB 00:00 2024-04-25T00:58:49.365 INFO:teuthology.orchestra.run.smithi171.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-04-25T00:58:49.440 INFO:teuthology.orchestra.run.smithi171.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-04-25T00:58:49.474 INFO:teuthology.orchestra.run.smithi171.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-25T00:58:49.508 INFO:teuthology.orchestra.run.smithi171.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-04-25T00:58:49.583 INFO:teuthology.orchestra.run.smithi171.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T00:58:49.608 INFO:teuthology.orchestra.run.smithi171.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-04-25T00:58:49.900 INFO:teuthology.orchestra.run.smithi171.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 636 kB/s | 292 kB 00:00 2024-04-25T00:58:50.480 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:58:50.480 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:58:50.491 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:58:50.492 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:58:50.493 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:58:50.494 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:58:50.495 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:58:50.497 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:58:50.498 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:58:50.557 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-25T00:58:50.566 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-25T00:58:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-04-25T00:58:50.567 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-25T00:58:50.567 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-04-25T00:58:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:58:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:58:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:58:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:58:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:58:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:58:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:58:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:58:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:58:50.569 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:58:50.569 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:58:50.569 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:58:50.569 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:58:50.569 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:58:50.570 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:58:50.570 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:58:50.570 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:58:50.570 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:58:50.570 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:58:50.570 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:58:50.571 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:58:50.572 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:58:50.573 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:58:50.574 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:58:50.575 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:58:50.576 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:58:50.577 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:58:50.578 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:58:50.579 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:58:50.580 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:58:50.581 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:58:50.582 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-04-25T00:58:50.583 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 4 Packages 2024-04-25T00:58:50.584 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:58:50.584 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 220 M 2024-04-25T00:58:50.585 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-04-25T00:58:50.925 INFO:teuthology.orchestra.run.smithi171.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-04-25T00:58:51.364 INFO:teuthology.orchestra.run.smithi129.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T00:58:51.564 INFO:teuthology.orchestra.run.smithi129.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 4.4 MB/s | 890 kB 00:00 2024-04-25T00:58:51.614 INFO:teuthology.orchestra.run.smithi129.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.8 MB/s | 145 kB 00:00 2024-04-25T00:58:51.864 INFO:teuthology.orchestra.run.smithi129.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 9.4 MB/s | 2.4 MB 00:00 2024-04-25T00:58:51.940 INFO:teuthology.orchestra.run.smithi129.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 7.2 MB/s | 5.4 MB 00:00 2024-04-25T00:58:52.034 INFO:teuthology.orchestra.run.smithi171.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.9 MB/s | 4.7 MB 00:02 2024-04-25T00:58:52.048 INFO:teuthology.orchestra.run.smithi129.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-25T00:58:52.159 INFO:teuthology.orchestra.run.smithi171.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 720 kB/s | 1.6 MB 00:02 2024-04-25T00:58:52.267 INFO:teuthology.orchestra.run.smithi171.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 679 kB/s | 911 kB 00:01 2024-04-25T00:58:52.332 INFO:teuthology.orchestra.run.smithi129.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-25T00:58:52.376 INFO:teuthology.orchestra.run.smithi171.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 390 kB/s | 133 kB 00:00 2024-04-25T00:58:52.568 INFO:teuthology.orchestra.run.smithi171.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-04-25T00:58:53.257 INFO:teuthology.orchestra.run.smithi129.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 11 MB/s | 10 MB 00:00 2024-04-25T00:58:53.324 INFO:teuthology.orchestra.run.smithi129.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 377 kB/s | 25 kB 00:00 2024-04-25T00:58:53.507 INFO:teuthology.orchestra.run.smithi129.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 12 MB/s | 17 MB 00:01 2024-04-25T00:58:53.574 INFO:teuthology.orchestra.run.smithi129.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.3 MB/s | 222 kB 00:00 2024-04-25T00:58:53.641 INFO:teuthology.orchestra.run.smithi129.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 405 kB/s | 27 kB 00:00 2024-04-25T00:58:53.660 INFO:teuthology.orchestra.run.smithi171.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 742 kB/s | 1.0 MB 00:01 2024-04-25T00:58:53.691 INFO:teuthology.orchestra.run.smithi129.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 636 kB/s | 32 kB 00:00 2024-04-25T00:58:53.792 INFO:teuthology.orchestra.run.smithi129.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 7.3 MB/s | 745 kB 00:00 2024-04-25T00:58:53.858 INFO:teuthology.orchestra.run.smithi129.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-25T00:58:53.925 INFO:teuthology.orchestra.run.smithi129.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-25T00:58:54.002 INFO:teuthology.orchestra.run.smithi171.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 715 kB/s | 244 kB 00:00 2024-04-25T00:58:54.011 INFO:teuthology.orchestra.run.smithi129.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.9 MB/s | 501 kB 00:00 2024-04-25T00:58:54.352 INFO:teuthology.orchestra.run.smithi171.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 708 kB/s | 248 kB 00:00 2024-04-25T00:58:54.460 INFO:teuthology.orchestra.run.smithi171.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-04-25T00:58:54.509 INFO:teuthology.orchestra.run.smithi129.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 5.3 MB 00:00 2024-04-25T00:58:54.561 INFO:teuthology.orchestra.run.smithi171.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-04-25T00:58:54.576 INFO:teuthology.orchestra.run.smithi129.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 675 kB/s | 45 kB 00:00 2024-04-25T00:58:54.652 INFO:teuthology.orchestra.run.smithi129.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-25T00:58:54.710 INFO:teuthology.orchestra.run.smithi129.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.7 MB/s | 162 kB 00:00 2024-04-25T00:58:54.785 INFO:teuthology.orchestra.run.smithi129.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 4.2 MB/s | 321 kB 00:00 2024-04-25T00:58:54.828 INFO:teuthology.orchestra.run.smithi171.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 646 kB/s | 172 kB 00:00 2024-04-25T00:58:54.861 INFO:teuthology.orchestra.run.smithi129.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 299 kB 00:00 2024-04-25T00:58:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.7 MB/s | 99 kB 00:00 2024-04-25T00:58:54.936 INFO:teuthology.orchestra.run.smithi171.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-25T00:58:54.978 INFO:teuthology.orchestra.run.smithi129.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T00:58:55.287 INFO:teuthology.orchestra.run.smithi129.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 10 MB/s | 3.1 MB 00:00 2024-04-25T00:58:55.345 INFO:teuthology.orchestra.run.smithi129.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-25T00:58:55.404 INFO:teuthology.orchestra.run.smithi129.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 546 kB/s | 32 kB 00:00 2024-04-25T00:58:55.471 INFO:teuthology.orchestra.run.smithi129.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.1 MB/s | 141 kB 00:00 2024-04-25T00:58:55.830 INFO:teuthology.orchestra.run.smithi129.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 9.8 MB/s | 3.5 MB 00:00 2024-04-25T00:58:56.547 INFO:teuthology.orchestra.run.smithi129.stdout:(31/152): ceph-common-19.0.0-2445.gd2c45d97.el9 3.9 MB/s | 21 MB 00:05 2024-04-25T00:58:56.639 INFO:teuthology.orchestra.run.smithi129.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 9.1 MB/s | 7.4 MB 00:00 2024-04-25T00:58:56.664 INFO:teuthology.orchestra.run.smithi129.stdout:(33/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.1 MB/s | 249 kB 00:00 2024-04-25T00:58:56.689 INFO:teuthology.orchestra.run.smithi129.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 987 kB/s | 49 kB 00:00 2024-04-25T00:58:57.106 INFO:teuthology.orchestra.run.smithi129.stdout:(35/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 13 MB/s | 48 MB 00:03 2024-04-25T00:58:57.131 INFO:teuthology.orchestra.run.smithi129.stdout:(36/152): ceph-prometheus-alerts-19.0.0-2445.gd 33 kB/s | 15 kB 00:00 2024-04-25T00:58:57.156 INFO:teuthology.orchestra.run.smithi129.stdout:(37/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 564 kB/s | 264 kB 00:00 2024-04-25T00:58:57.190 INFO:teuthology.orchestra.run.smithi129.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 313 kB/s | 18 kB 00:00 2024-04-25T00:58:57.215 INFO:teuthology.orchestra.run.smithi129.stdout:(39/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-25T00:58:57.307 INFO:teuthology.orchestra.run.smithi129.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 534 kB/s | 80 kB 00:00 2024-04-25T00:58:57.399 INFO:teuthology.orchestra.run.smithi129.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-25T00:58:57.424 INFO:teuthology.orchestra.run.smithi129.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-25T00:58:57.483 INFO:teuthology.orchestra.run.smithi129.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T00:58:57.517 INFO:teuthology.orchestra.run.smithi129.stdout:(44/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-25T00:58:57.542 INFO:teuthology.orchestra.run.smithi129.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T00:58:57.575 INFO:teuthology.orchestra.run.smithi129.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T00:58:57.609 INFO:teuthology.orchestra.run.smithi129.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-04-25T00:58:57.635 INFO:teuthology.orchestra.run.smithi129.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T00:58:58.060 INFO:teuthology.orchestra.run.smithi129.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-04-25T00:58:58.169 INFO:teuthology.orchestra.run.smithi129.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-25T00:58:58.352 INFO:teuthology.orchestra.run.smithi129.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 888 kB/s | 689 kB 00:00 2024-04-25T00:58:58.394 INFO:teuthology.orchestra.run.smithi129.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-25T00:58:58.453 INFO:teuthology.orchestra.run.smithi129.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-04-25T00:58:58.478 INFO:teuthology.orchestra.run.smithi129.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-04-25T00:58:58.561 INFO:teuthology.orchestra.run.smithi129.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-25T00:58:58.586 INFO:teuthology.orchestra.run.smithi129.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-25T00:58:58.662 INFO:teuthology.orchestra.run.smithi129.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-25T00:58:58.687 INFO:teuthology.orchestra.run.smithi129.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-25T00:58:58.887 INFO:teuthology.orchestra.run.smithi129.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-04-25T00:58:58.929 INFO:teuthology.orchestra.run.smithi129.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-04-25T00:58:58.954 INFO:teuthology.orchestra.run.smithi129.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-04-25T00:58:58.980 INFO:teuthology.orchestra.run.smithi129.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-04-25T00:58:59.022 INFO:teuthology.orchestra.run.smithi129.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-04-25T00:58:59.063 INFO:teuthology.orchestra.run.smithi129.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-25T00:58:59.089 INFO:teuthology.orchestra.run.smithi129.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T00:58:59.114 INFO:teuthology.orchestra.run.smithi129.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-04-25T00:58:59.389 INFO:teuthology.orchestra.run.smithi129.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-04-25T00:58:59.515 INFO:teuthology.orchestra.run.smithi129.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-04-25T00:58:59.623 INFO:teuthology.orchestra.run.smithi129.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T00:58:59.953 INFO:teuthology.orchestra.run.smithi171.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 759 kB/s | 5.8 MB 00:07 2024-04-25T00:59:00.303 INFO:teuthology.orchestra.run.smithi171.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-04-25T00:59:00.653 INFO:teuthology.orchestra.run.smithi171.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-04-25T00:59:00.728 INFO:teuthology.orchestra.run.smithi171.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 748 kB/s | 6.0 MB 00:08 2024-04-25T00:59:00.854 INFO:teuthology.orchestra.run.smithi171.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 338 kB/s | 42 kB 00:00 2024-04-25T00:59:00.879 INFO:teuthology.orchestra.run.smithi171.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 342 kB/s | 77 kB 00:00 2024-04-25T00:59:01.146 INFO:teuthology.orchestra.run.smithi171.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 596 kB/s | 159 kB 00:00 2024-04-25T00:59:01.279 INFO:teuthology.orchestra.run.smithi171.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 633 kB/s | 269 kB 00:00 2024-04-25T00:59:01.380 INFO:teuthology.orchestra.run.smithi171.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-04-25T00:59:01.423 INFO:teuthology.orchestra.run.smithi129.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.0 MB/s | 4.7 MB 00:02 2024-04-25T00:59:01.505 INFO:teuthology.orchestra.run.smithi171.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-04-25T00:59:01.614 INFO:teuthology.orchestra.run.smithi171.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 596 kB/s | 279 kB 00:00 2024-04-25T00:59:01.764 INFO:teuthology.orchestra.run.smithi171.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 764 kB/s | 5.1 MB 00:06 2024-04-25T00:59:01.781 INFO:teuthology.orchestra.run.smithi171.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 249 kB/s | 42 kB 00:00 2024-04-25T00:59:01.906 INFO:teuthology.orchestra.run.smithi171.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 311 kB/s | 44 kB 00:00 2024-04-25T00:59:02.006 INFO:teuthology.orchestra.run.smithi171.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-04-25T00:59:02.115 INFO:teuthology.orchestra.run.smithi171.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-25T00:59:02.182 INFO:teuthology.orchestra.run.smithi171.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 551 kB/s | 221 kB 00:00 2024-04-25T00:59:02.299 INFO:teuthology.orchestra.run.smithi171.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 495 kB/s | 58 kB 00:00 2024-04-25T00:59:02.516 INFO:teuthology.orchestra.run.smithi171.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 433 kB/s | 94 kB 00:00 2024-04-25T00:59:02.616 INFO:teuthology.orchestra.run.smithi171.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 609 kB/s | 305 kB 00:00 2024-04-25T00:59:02.733 INFO:teuthology.orchestra.run.smithi171.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-04-25T00:59:02.750 INFO:teuthology.orchestra.run.smithi171.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 272 kB/s | 64 kB 00:00 2024-04-25T00:59:02.823 INFO:teuthology.orchestra.run.smithi129.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-04-25T00:59:02.950 INFO:teuthology.orchestra.run.smithi171.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-04-25T00:59:03.023 INFO:teuthology.orchestra.run.smithi129.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 646 kB/s | 1.0 MB 00:01 2024-04-25T00:59:03.049 INFO:teuthology.orchestra.run.smithi129.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 389 kB/s | 87 kB 00:00 2024-04-25T00:59:03.100 INFO:teuthology.orchestra.run.smithi171.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.6 MB/s | 548 kB 00:00 2024-04-25T00:59:03.150 INFO:teuthology.orchestra.run.smithi171.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-04-25T00:59:03.157 INFO:teuthology.orchestra.run.smithi129.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-25T00:59:03.176 INFO:teuthology.orchestra.run.smithi171.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-25T00:59:03.235 INFO:teuthology.orchestra.run.smithi171.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-25T00:59:03.276 INFO:teuthology.orchestra.run.smithi171.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 624 kB/s | 26 kB 00:00 2024-04-25T00:59:03.282 INFO:teuthology.orchestra.run.smithi129.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-04-25T00:59:03.319 INFO:teuthology.orchestra.run.smithi171.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-25T00:59:03.369 INFO:teuthology.orchestra.run.smithi171.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T00:59:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-04-25T00:59:03.428 INFO:teuthology.orchestra.run.smithi171.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T00:59:03.453 INFO:teuthology.orchestra.run.smithi171.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-25T00:59:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-25T00:59:03.494 INFO:teuthology.orchestra.run.smithi171.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-25T00:59:03.528 INFO:teuthology.orchestra.run.smithi171.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T00:59:03.545 INFO:teuthology.orchestra.run.smithi171.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-04-25T00:59:03.562 INFO:teuthology.orchestra.run.smithi171.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 959 kB/s | 32 kB 00:00 2024-04-25T00:59:03.579 INFO:teuthology.orchestra.run.smithi171.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 449 kB/s | 15 kB 00:00 2024-04-25T00:59:03.592 INFO:teuthology.orchestra.run.smithi129.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-25T00:59:03.604 INFO:teuthology.orchestra.run.smithi171.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-04-25T00:59:03.639 INFO:teuthology.orchestra.run.smithi171.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T00:59:03.663 INFO:teuthology.orchestra.run.smithi171.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-04-25T00:59:03.689 INFO:teuthology.orchestra.run.smithi171.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 704 kB/s | 35 kB 00:00 2024-04-25T00:59:03.700 INFO:teuthology.orchestra.run.smithi129.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-25T00:59:03.722 INFO:teuthology.orchestra.run.smithi171.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T00:59:03.772 INFO:teuthology.orchestra.run.smithi171.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-25T00:59:03.823 INFO:teuthology.orchestra.run.smithi171.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-25T00:59:03.848 INFO:teuthology.orchestra.run.smithi171.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-25T00:59:03.873 INFO:teuthology.orchestra.run.smithi171.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-25T00:59:03.899 INFO:teuthology.orchestra.run.smithi171.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-25T00:59:03.924 INFO:teuthology.orchestra.run.smithi171.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-25T00:59:03.949 INFO:teuthology.orchestra.run.smithi171.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-25T00:59:04.000 INFO:teuthology.orchestra.run.smithi171.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-25T00:59:04.025 INFO:teuthology.orchestra.run.smithi171.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-04-25T00:59:04.042 INFO:teuthology.orchestra.run.smithi171.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-04-25T00:59:04.067 INFO:teuthology.orchestra.run.smithi171.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-04-25T00:59:04.092 INFO:teuthology.orchestra.run.smithi171.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-25T00:59:04.118 INFO:teuthology.orchestra.run.smithi171.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-25T00:59:04.143 INFO:teuthology.orchestra.run.smithi171.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-25T00:59:04.168 INFO:teuthology.orchestra.run.smithi171.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-25T00:59:04.193 INFO:teuthology.orchestra.run.smithi171.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-25T00:59:04.218 INFO:teuthology.orchestra.run.smithi171.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-25T00:59:04.244 INFO:teuthology.orchestra.run.smithi171.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-04-25T00:59:04.278 INFO:teuthology.orchestra.run.smithi171.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-04-25T00:59:04.303 INFO:teuthology.orchestra.run.smithi171.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 873 kB/s | 51 kB 00:00 2024-04-25T00:59:04.336 INFO:teuthology.orchestra.run.smithi171.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-04-25T00:59:04.362 INFO:teuthology.orchestra.run.smithi171.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T00:59:04.387 INFO:teuthology.orchestra.run.smithi171.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-25T00:59:04.412 INFO:teuthology.orchestra.run.smithi171.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-25T00:59:04.438 INFO:teuthology.orchestra.run.smithi171.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T00:59:04.546 INFO:teuthology.orchestra.run.smithi171.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T00:59:05.192 INFO:teuthology.orchestra.run.smithi129.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-04-25T00:59:05.438 INFO:teuthology.orchestra.run.smithi171.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-04-25T00:59:05.488 INFO:teuthology.orchestra.run.smithi171.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.2 MB/s | 3.4 MB 00:01 2024-04-25T00:59:05.526 INFO:teuthology.orchestra.run.smithi129.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 752 kB/s | 250 kB 00:00 2024-04-25T00:59:05.884 INFO:teuthology.orchestra.run.smithi129.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 620 kB/s | 222 kB 00:00 2024-04-25T00:59:06.068 INFO:teuthology.orchestra.run.smithi129.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-04-25T00:59:06.168 INFO:teuthology.orchestra.run.smithi129.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-04-25T00:59:06.280 INFO:teuthology.orchestra.run.smithi171.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-25T00:59:06.489 INFO:teuthology.orchestra.run.smithi171.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-04-25T00:59:06.602 INFO:teuthology.orchestra.run.smithi129.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 621 kB/s | 269 kB 00:00 2024-04-25T00:59:06.877 INFO:teuthology.orchestra.run.smithi129.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 578 kB/s | 159 kB 00:00 2024-04-25T00:59:07.294 INFO:teuthology.orchestra.run.smithi129.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 669 kB/s | 279 kB 00:00 2024-04-25T00:59:07.394 INFO:teuthology.orchestra.run.smithi129.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-04-25T00:59:07.494 INFO:teuthology.orchestra.run.smithi129.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-25T00:59:08.012 INFO:teuthology.orchestra.run.smithi129.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.2 MB/s | 5.1 MB 00:04 2024-04-25T00:59:08.112 INFO:teuthology.orchestra.run.smithi129.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 418 kB/s | 42 kB 00:00 2024-04-25T00:59:08.212 INFO:teuthology.orchestra.run.smithi129.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-04-25T00:59:08.554 INFO:teuthology.orchestra.run.smithi129.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 646 kB/s | 221 kB 00:00 2024-04-25T00:59:08.654 INFO:teuthology.orchestra.run.smithi129.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-04-25T00:59:08.754 INFO:teuthology.orchestra.run.smithi129.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 475 kB/s | 47 kB 00:00 2024-04-25T00:59:09.171 INFO:teuthology.orchestra.run.smithi129.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 731 kB/s | 305 kB 00:00 2024-04-25T00:59:09.280 INFO:teuthology.orchestra.run.smithi129.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-04-25T00:59:09.456 INFO:teuthology.orchestra.run.smithi129.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 536 kB/s | 94 kB 00:00 2024-04-25T00:59:09.644 INFO:teuthology.orchestra.run.smithi129.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-04-25T00:59:09.739 INFO:teuthology.orchestra.run.smithi129.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-04-25T00:59:09.818 INFO:teuthology.orchestra.run.smithi129.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-25T00:59:09.957 INFO:teuthology.orchestra.run.smithi129.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-25T00:59:10.240 INFO:teuthology.orchestra.run.smithi129.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-25T00:59:10.307 INFO:teuthology.orchestra.run.smithi129.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-04-25T00:59:10.390 INFO:teuthology.orchestra.run.smithi129.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-25T00:59:10.649 INFO:teuthology.orchestra.run.smithi129.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T00:59:10.716 INFO:teuthology.orchestra.run.smithi129.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-04-25T00:59:10.785 INFO:teuthology.orchestra.run.smithi129.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 707 kB/s | 49 kB 00:00 2024-04-25T00:59:10.827 INFO:teuthology.orchestra.run.smithi129.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T00:59:10.885 INFO:teuthology.orchestra.run.smithi129.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T00:59:10.969 INFO:teuthology.orchestra.run.smithi129.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-25T00:59:11.044 INFO:teuthology.orchestra.run.smithi129.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-04-25T00:59:11.103 INFO:teuthology.orchestra.run.smithi129.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 745 kB/s | 43 kB 00:00 2024-04-25T00:59:11.220 INFO:teuthology.orchestra.run.smithi129.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-25T00:59:11.270 INFO:teuthology.orchestra.run.smithi129.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-25T00:59:11.338 INFO:teuthology.orchestra.run.smithi129.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T00:59:11.404 INFO:teuthology.orchestra.run.smithi129.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-25T00:59:11.480 INFO:teuthology.orchestra.run.smithi129.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-25T00:59:11.538 INFO:teuthology.orchestra.run.smithi129.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 608 kB/s | 35 kB 00:00 2024-04-25T00:59:11.647 INFO:teuthology.orchestra.run.smithi129.stdout:(120/152): python3-babel-2.9.1-2.el9.noarch.rpm 708 kB/s | 6.0 MB 00:08 2024-04-25T00:59:11.672 INFO:teuthology.orchestra.run.smithi129.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-25T00:59:11.739 INFO:teuthology.orchestra.run.smithi129.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-25T00:59:11.806 INFO:teuthology.orchestra.run.smithi129.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-25T00:59:11.856 INFO:teuthology.orchestra.run.smithi129.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-04-25T00:59:11.915 INFO:teuthology.orchestra.run.smithi129.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-04-25T00:59:11.965 INFO:teuthology.orchestra.run.smithi129.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-25T00:59:12.015 INFO:teuthology.orchestra.run.smithi129.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-25T00:59:12.057 INFO:teuthology.orchestra.run.smithi129.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-04-25T00:59:12.124 INFO:teuthology.orchestra.run.smithi129.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-25T00:59:12.149 INFO:teuthology.orchestra.run.smithi129.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-04-25T00:59:12.183 INFO:teuthology.orchestra.run.smithi129.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T00:59:12.208 INFO:teuthology.orchestra.run.smithi129.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 982 kB/s | 58 kB 00:00 2024-04-25T00:59:12.250 INFO:teuthology.orchestra.run.smithi129.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-25T00:59:12.276 INFO:teuthology.orchestra.run.smithi129.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-04-25T00:59:12.309 INFO:teuthology.orchestra.run.smithi129.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-25T00:59:12.334 INFO:teuthology.orchestra.run.smithi129.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-25T00:59:12.377 INFO:teuthology.orchestra.run.smithi129.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-04-25T00:59:12.402 INFO:teuthology.orchestra.run.smithi129.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-25T00:59:12.435 INFO:teuthology.orchestra.run.smithi129.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T00:59:12.461 INFO:teuthology.orchestra.run.smithi129.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 609 kB/s | 36 kB 00:00 2024-04-25T00:59:12.486 INFO:teuthology.orchestra.run.smithi129.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T00:59:12.536 INFO:teuthology.orchestra.run.smithi129.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T00:59:12.553 INFO:teuthology.orchestra.run.smithi129.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-25T00:59:12.595 INFO:teuthology.orchestra.run.smithi129.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-25T00:59:12.621 INFO:teuthology.orchestra.run.smithi129.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-25T00:59:12.717 INFO:teuthology.orchestra.run.smithi129.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-25T00:59:12.771 INFO:teuthology.orchestra.run.smithi129.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T00:59:13.038 INFO:teuthology.orchestra.run.smithi129.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 11 MB/s | 3.4 MB 00:00 2024-04-25T00:59:13.122 INFO:teuthology.orchestra.run.smithi129.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-25T00:59:13.330 INFO:teuthology.orchestra.run.smithi129.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-04-25T00:59:13.464 INFO:teuthology.orchestra.run.smithi129.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-04-25T00:59:23.213 INFO:teuthology.orchestra.run.smithi171.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 767 kB/s | 16 MB 00:21 2024-04-25T00:59:23.215 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:59:23.216 INFO:teuthology.orchestra.run.smithi171.stdout:Total 4.8 MB/s | 220 MB 00:46 2024-04-25T00:59:24.313 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-25T00:59:24.459 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-25T00:59:24.459 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-25T00:59:26.228 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-25T00:59:26.228 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-25T00:59:28.896 INFO:teuthology.orchestra.run.smithi129.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 778 kB/s | 16 MB 00:21 2024-04-25T00:59:28.899 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:59:28.899 INFO:teuthology.orchestra.run.smithi129.stdout:Total 5.7 MB/s | 220 MB 00:38 2024-04-25T00:59:30.019 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-04-25T00:59:30.160 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-04-25T00:59:30.161 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-04-25T00:59:31.913 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-04-25T00:59:31.914 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-04-25T00:59:32.172 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-25T00:59:32.249 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:59:32.287 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:59:32.340 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:59:32.374 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:59:32.400 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:59:32.443 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:59:32.790 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:59:32.910 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:59:33.031 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:59:33.061 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:59:33.160 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:59:33.190 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:59:33.344 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:59:33.395 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:59:33.423 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:59:33.526 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:59:33.549 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:59:33.611 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:59:33.633 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:59:33.652 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:59:33.867 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:59:33.900 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:59:34.007 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:59:34.095 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:59:34.140 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:59:34.175 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:59:34.240 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:59:34.282 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:59:34.326 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:59:34.358 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:59:34.420 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:59:34.488 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:59:34.542 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:59:34.573 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:59:34.599 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:59:34.675 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:59:34.710 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:59:34.754 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:59:34.782 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:59:34.853 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:59:34.897 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:59:34.972 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:59:35.109 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:59:35.166 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:59:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:59:35.779 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:59:35.808 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:59:35.850 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:59:35.888 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:59:35.930 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:59:36.103 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:59:36.158 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:59:36.213 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:59:36.287 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:59:36.317 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:59:36.354 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:59:36.401 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:59:36.472 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:59:36.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:59:36.764 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:59:36.902 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:59:36.953 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:59:36.983 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:59:37.024 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:59:37.119 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:59:37.158 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:59:37.198 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:59:37.223 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:59:37.248 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:59:37.324 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:59:37.450 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:59:37.481 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:59:37.509 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:59:37.543 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:59:37.582 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:59:37.613 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:59:37.643 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:59:37.686 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:59:37.720 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:59:37.761 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:59:37.796 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:59:37.832 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-04-25T00:59:37.883 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:59:37.924 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:59:37.926 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:59:37.957 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:59:37.963 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:59:38.004 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:59:38.024 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:59:38.059 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:59:38.085 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:59:38.104 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:59:38.126 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:59:38.159 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:59:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:59:38.552 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:59:38.673 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:59:38.703 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:59:38.777 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:59:38.815 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:59:38.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:59:38.948 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:59:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:59:38.982 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:59:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:59:39.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:59:39.109 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:59:39.143 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:59:39.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:59:39.173 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:59:39.229 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:59:39.250 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:59:39.262 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:59:39.272 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:59:39.293 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:59:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:59:39.471 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:59:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:59:39.498 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:59:39.576 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:59:39.598 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:59:39.694 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:59:39.724 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:59:39.758 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:59:39.822 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:59:39.857 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:59:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:59:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:59:39.970 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:59:40.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:59:40.092 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:59:40.123 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:59:40.158 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:59:40.231 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:59:40.233 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:59:40.268 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:59:40.303 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:59:40.332 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:59:40.347 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:59:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:59:40.447 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:59:40.521 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:59:40.652 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:59:40.699 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:59:41.229 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:59:41.311 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:59:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:59:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:59:41.421 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:59:41.463 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:59:41.605 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:59:41.627 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:59:41.694 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:59:41.699 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:59:41.722 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:59:41.753 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:59:41.755 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:59:41.817 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:59:41.875 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:59:42.028 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:59:42.066 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:59:42.110 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:59:42.183 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:59:42.209 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:59:42.219 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:59:42.239 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:59:42.262 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:59:42.276 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:59:42.304 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:59:42.439 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:59:42.484 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:59:42.515 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:59:42.556 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:59:42.649 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:59:42.690 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:59:42.729 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:59:42.746 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:59:42.755 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:59:42.771 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:59:42.780 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:59:42.842 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:59:42.854 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:59:42.873 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:59:42.981 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:59:43.012 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:59:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:59:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:59:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:59:43.145 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:59:43.175 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:59:43.209 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:59:43.243 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:59:43.284 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:59:43.319 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:59:43.413 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:59:43.466 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:59:43.497 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:59:43.544 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:59:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:59:43.689 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:59:44.413 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:59:44.454 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:59:44.487 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:59:44.584 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:59:44.637 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:59:44.723 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:59:44.773 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:59:44.967 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:59:44.996 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:59:45.073 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:59:45.119 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:59:45.161 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:59:45.699 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:59:45.712 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:59:45.756 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:59:45.827 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:59:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:59:45.847 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:59:45.847 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:59:45.847 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:59:45.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:59:47.074 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:59:47.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:59:47.194 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:59:47.226 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:59:47.538 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:59:47.581 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:59:47.654 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:59:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:59:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:59:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:59:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:59:48.288 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:59:48.311 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:59:50.620 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:59:50.657 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:59:51.198 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:59:51.252 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:59:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:59:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:59:51.343 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:59:51.343 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T00:59:51.370 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:59:57.579 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:59:57.579 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-04-25T00:59:57.579 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-04-25T00:59:57.579 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-04-25T00:59:57.580 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-04-25T00:59:57.580 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-04-25T00:59:57.580 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-04-25T00:59:57.580 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-04-25T00:59:57.580 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:59:57.745 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:59:57.762 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:59:57.762 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:59:57.762 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:59:57.762 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:59:59.274 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:59:59.299 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:59:59.299 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:59:59.299 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:59:59.299 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:59:59.300 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:59:59.300 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:59:59.532 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:59:59.554 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:59:59.554 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:59:59.555 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:59:59.555 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:59:59.555 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:59:59.555 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T00:59:59.991 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:00:00.030 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:00:00.030 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:00.031 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:00:00.031 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:00:00.031 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:00:00.031 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:00.138 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:00:00.187 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:00:00.242 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:00:00.438 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:00:00.492 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:00:00.571 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:00:00.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:00:00.807 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:00:00.842 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:00:00.882 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:00:00.919 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:00:01.326 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:00:01.350 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:00:01.382 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:00:01.435 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:00:01.465 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:00:02.498 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:00:02.542 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-04-25T01:00:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:02.981 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:00:02.997 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:00:02.998 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:02.998 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:00:02.998 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:03.509 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:00:03.542 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:00:03.645 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:00:03.808 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:00:03.844 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:00:03.868 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:00:03.868 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:03.868 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:00:03.868 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:00:03.868 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:00:03.868 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:03.912 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:00:03.923 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:00:04.021 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:00:04.069 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:00:04.476 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:00:04.502 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:00:04.503 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:04.503 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:00:04.503 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:00:04.503 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:00:04.503 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:00:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:00:04.750 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:04.750 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:00:04.750 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:00:04.750 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:00:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:04.963 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:00:04.989 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:00:05.011 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:00:05.012 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:05.012 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:00:05.012 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:00:05.012 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:00:05.012 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:05.057 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:00:05.079 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:00:05.079 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:05.079 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:00:05.079 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:05.132 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:00:05.154 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:00:05.154 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:05.154 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:00:05.154 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:00:05.154 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:00:05.155 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:05.194 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:00:05.216 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:00:05.217 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:05.217 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:00:05.217 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:00:05.217 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:00:05.217 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:05.315 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:00:05.355 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:00:05.411 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:00:05.458 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:00:05.480 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:00:05.480 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:05.480 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:00:05.480 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:00:05.480 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:00:05.480 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:05.596 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:00:05.636 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:00:05.706 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:00:05.794 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:00:05.932 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:00:05.969 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:00:06.001 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:00:06.029 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:00:06.478 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:00:06.502 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:00:06.534 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:00:06.596 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:00:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:00:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:00:07.702 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:00:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:00:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:00:08.795 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:00:08.956 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:00:08.987 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:00:09.011 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:00:09.011 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:09.011 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:00:09.011 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:00:09.011 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:00:09.011 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:09.055 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:00:09.066 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:00:09.164 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:00:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:00:10.012 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:00:10.053 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:00:10.094 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:00:10.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:00:10.126 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:00:10.149 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:00:10.153 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:00:10.172 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:00:10.172 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:10.172 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:00:10.173 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:00:10.173 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:00:10.173 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:10.217 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:00:10.239 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:00:10.239 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:10.239 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:00:10.239 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:10.284 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:00:10.306 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:00:10.306 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:10.307 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:00:10.307 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:00:10.307 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:00:10.307 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:10.342 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:00:10.382 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:00:10.412 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:00:10.609 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:00:10.632 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:00:10.632 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:00:10.632 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:00:10.632 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:00:10.633 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:00:10.633 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:11.039 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:00:11.085 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:00:11.105 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:00:11.955 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:00:12.049 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:00:12.090 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:00:12.173 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:00:12.202 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:00:12.202 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:00:12.288 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:00:12.313 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:00:12.313 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:00:12.419 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:00:12.486 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:00:15.213 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:00:15.254 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:00:15.295 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:00:15.327 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:00:15.355 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:00:15.527 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:00:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:00:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:00:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:00:16.269 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:00:16.289 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:00:17.110 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:00:17.198 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:00:17.240 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:00:17.321 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:00:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:00:17.394 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:00:17.470 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:00:17.503 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:00:17.503 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:00:17.593 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:00:17.659 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:00:18.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:00:18.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:00:18.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:00:18.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:00:18.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:00:18.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:00:18.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:00:18.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:00:18.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:00:18.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:00:18.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:00:18.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:00:18.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:00:18.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:00:18.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:00:18.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:00:18.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:00:18.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:00:20.618 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:00:20.618 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:20.618 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-04-25T01:00:20.618 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.619 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.619 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:00:20.619 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:00:20.619 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-04-25T01:00:20.619 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:00:20.619 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:00:20.620 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:00:20.620 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:00:20.620 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.620 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.620 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.620 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.621 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.622 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.622 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.622 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.622 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.622 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.622 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.623 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.623 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.623 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.623 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.623 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.623 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.624 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.624 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.624 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:20.624 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:00:20.624 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:00:20.624 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:00:20.625 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:00:20.625 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:00:20.625 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:00:20.625 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:00:20.625 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:00:20.625 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:00:20.626 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:00:20.626 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:00:20.626 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.626 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.626 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.627 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:00:20.628 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:00:20.629 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:00:20.630 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:00:20.630 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:00:20.630 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:00:20.630 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:00:20.630 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:00:20.630 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.631 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:00:20.632 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:00:20.633 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:00:20.634 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:00:20.635 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:00:20.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:00:20.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:00:20.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:00:20.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:00:20.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:00:20.636 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:00:20.637 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:00:20.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:00:20.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:00:20.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:00:20.638 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:00:20.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:00:20.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:00:20.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:00:20.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:00:20.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:00:20.639 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.640 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.641 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:20.641 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:00:20.641 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:00:20.641 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:00:20.641 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:00:20.641 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:00:20.642 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:00:20.642 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:00:20.642 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:20.642 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-25T01:00:20.998 DEBUG:teuthology.parallel:result is None 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:00:23.775 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:00:23.776 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:00:23.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:00:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:00:23.779 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:00:23.780 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:00:23.781 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:00:23.782 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:00:23.783 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:00:23.784 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:00:23.785 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:00:25.773 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:00:25.773 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:25.773 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-04-25T01:00:25.773 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.773 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.774 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.775 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:00:25.776 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:00:25.777 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:00:25.778 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:00:25.779 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:00:25.780 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:00:25.781 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.782 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:00:25.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:00:25.784 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:00:25.785 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:00:25.785 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:25.785 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-25T01:00:26.159 DEBUG:teuthology.parallel:result is None 2024-04-25T01:00:26.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:00:26.381 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:00:26.403 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:00:26.403 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:00:26.403 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:00:26.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:00:26.544 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:00:26.571 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:00:26.571 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:00:26.571 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:00:26.574 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T01:00:26.574 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:26.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:00:26.601 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:26.601 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:00:26.642 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T01:00:26.642 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:26.642 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:00:26.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:00:26.730 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:26.730 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:00:26.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:00:26.827 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T01:00:26.828 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:26.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:00:26.854 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:00:26.919 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:26.919 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:00:26.947 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:00:27.015 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T01:00:27.016 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:27.016 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:00:27.039 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:00:27.103 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:27.104 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:00:27.131 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:00:27.198 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T01:00:27.282 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T01:00:27.283 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-04-25T01:00:27.285 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /var/log/ceph 2024-04-25T01:00:27.312 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T01:00:27.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T01:00:27.349 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T01:00:27.378 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T01:00:27.378 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T01:00:27.414 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T01:00:27.449 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T01:00:27.449 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T01:00:27.449 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T01:00:27.449 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:27.449 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T01:00:27.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T01:00:27.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T01:00:27.636 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:27.636 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T01:00:27.651 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T01:00:27.761 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T01:00:27.790 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T01:00:27.791 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T01:00:27.791 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302', 'branch': 'squid', 'description': 'fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672302', 'kclient-workload': {'sequential': [{'kclient': ['client.1']}, {'workunit': {'clients': {'client.1': ['suites/dbench.sh']}}}]}, 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'valgrind': {'exit_on_first_error': False}}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCVQ59ja3FdqN+8CNqzAlGvQtPuTBLf2jfZ8JYDtY3/Jgcfe4zv3FAMmEKWv8Z523cELqFpb0Ps/7/q7N6BK9Kc=', 'smithi171.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqSqd/60S1Qait7GDVjA8IWPelcig9o1NLqgKeHwnpIDWko8RhGZ/RcwxpmpFE4LEGx+speFRGVhoPK2tmR8hY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'valgrind': {'exit_on_first_error': False}, 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'parallel': ['user-workload', 'kclient-workload']}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'user-workload': {'sequential': [{'ceph-fuse': ['client.0']}, {'workunit': {'clients': {'client.0': ['suites/iozone.sh']}}}]}, 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T01:00:27.791 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T01:00:27.794 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T01:00:27.840 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T01:00:27.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T01:00:27.905 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:27.905 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:00:27.925 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:00:27.925 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-04-25T01:00:27.980 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:00:27.980 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:27.980 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T01:00:27.980 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:27.980 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:27.981 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 01:00:18.066534634 +0000 2024-04-25T01:00:27.981 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:57:52.074982724 +0000 2024-04-25T01:00:27.981 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:57:52.074982724 +0000 2024-04-25T01:00:27.981 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 00:57:52.074982724 +0000 2024-04-25T01:00:27.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:00:28.042 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T01:00:28.042 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T01:00:28.042 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.00019598 s, 2.6 MB/s 2024-04-25T01:00:28.043 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:00:28.099 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-04-25T01:00:28.154 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:00:28.154 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:28.154 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T01:00:28.154 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 01:00:18.066534634 +0000 2024-04-25T01:00:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:57:52.413975351 +0000 2024-04-25T01:00:28.155 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:57:52.413975351 +0000 2024-04-25T01:00:28.155 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 00:57:52.413975351 +0000 2024-04-25T01:00:28.155 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:00:28.216 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T01:00:28.216 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T01:00:28.216 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000171403 s, 3.0 MB/s 2024-04-25T01:00:28.217 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:00:28.273 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-04-25T01:00:28.327 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 01:00:18.066534634 +0000 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:57:52.749968044 +0000 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:57:52.749968044 +0000 2024-04-25T01:00:28.328 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 00:57:52.749968044 +0000 2024-04-25T01:00:28.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:00:28.390 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T01:00:28.390 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T01:00:28.390 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000172443 s, 3.0 MB/s 2024-04-25T01:00:28.391 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:00:28.447 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-04-25T01:00:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:00:28.502 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:28.502 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T01:00:28.503 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:28.503 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:28.503 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-25 01:00:18.067534612 +0000 2024-04-25T01:00:28.503 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 00:57:53.101960388 +0000 2024-04-25T01:00:28.503 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 00:57:53.101960388 +0000 2024-04-25T01:00:28.503 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-25 00:57:53.101960388 +0000 2024-04-25T01:00:28.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:00:28.564 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-25T01:00:28.564 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-25T01:00:28.564 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000166448 s, 3.1 MB/s 2024-04-25T01:00:28.565 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:00:28.621 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:28.621 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:28.621 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:00:28.638 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:00:28.638 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 01:00:12.854559447 +0000 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 00:57:52.265528157 +0000 2024-04-25T01:00:28.695 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 00:57:52.265528157 +0000 2024-04-25T01:00:28.696 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 00:57:52.265528157 +0000 2024-04-25T01:00:28.696 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:00:28.759 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T01:00:28.759 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T01:00:28.759 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000286758 s, 1.8 MB/s 2024-04-25T01:00:28.761 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:00:28.818 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-04-25T01:00:28.873 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:00:28.873 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:28.873 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T01:00:28.873 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:28.873 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:28.874 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 01:00:12.855559427 +0000 2024-04-25T01:00:28.874 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 00:57:52.629521110 +0000 2024-04-25T01:00:28.874 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 00:57:52.629521110 +0000 2024-04-25T01:00:28.874 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 00:57:52.629521110 +0000 2024-04-25T01:00:28.874 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:00:28.936 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T01:00:28.937 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T01:00:28.937 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000196787 s, 2.6 MB/s 2024-04-25T01:00:28.938 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:00:28.994 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 01:00:12.855559427 +0000 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 00:57:52.968514547 +0000 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 00:57:52.968514547 +0000 2024-04-25T01:00:29.050 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 00:57:52.968514547 +0000 2024-04-25T01:00:29.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:00:29.112 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T01:00:29.112 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T01:00:29.112 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000167135 s, 3.1 MB/s 2024-04-25T01:00:29.113 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:00:29.169 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-04-25T01:00:29.224 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:00:29.224 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:00:29.224 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T01:00:29.224 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:00:29.224 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:00:29.224 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 01:00:12.856559408 +0000 2024-04-25T01:00:29.225 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 00:57:53.333507481 +0000 2024-04-25T01:00:29.225 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 00:57:53.333507481 +0000 2024-04-25T01:00:29.225 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 00:57:53.333507481 +0000 2024-04-25T01:00:29.225 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:00:29.286 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T01:00:29.287 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T01:00:29.287 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000188854 s, 2.7 MB/s 2024-04-25T01:00:29.288 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:00:29.344 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:29.344 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T01:00:29.344 INFO:tasks.ceph:Generating config... 2024-04-25T01:00:29.345 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T01:00:29.345 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T01:00:29.345 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T01:00:29.345 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T01:00:29.346 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T01:00:29.346 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T01:00:29.346 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T01:00:29.347 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T01:00:29.347 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T01:00:29.347 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T01:00:29.347 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T01:00:29.347 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T01:00:29.347 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T01:00:29.347 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T01:00:29.347 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T01:00:29.347 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T01:00:29.347 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T01:00:29.347 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T01:00:29.348 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-25T01:00:29.348 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T01:00:29.348 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T01:00:29.413 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T01:00:29.415 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T01:00:29.456 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:00:29.521 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.129'), ('mon.b', '172.21.15.171'), ('mon.c', '[v2:172.21.15.171:3301,v1:172.21.15.171:6790]')] 2024-04-25T01:00:29.522 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.129,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T01:00:29.523 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:29.523 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T01:00:29.576 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.129 --add b 172.21.15.171 --addv c '[v2:172.21.15.171:3301,v1:172.21.15.171:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:29.660 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:29.660 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: generated fsid 4b43d0d1-0ff7-4061-ba99-dacfcc159ca4 2024-04-25T01:00:29.660 INFO:teuthology.orchestra.run.smithi129.stdout:setting min_mon_release = quincy 2024-04-25T01:00:29.660 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 0 2024-04-25T01:00:29.660 INFO:teuthology.orchestra.run.smithi129.stdout:fsid 4b43d0d1-0ff7-4061-ba99-dacfcc159ca4 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:last_changed 2024-04-25T01:00:29.661036+0000 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-04-25T01:00:29.661036+0000 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:min_mon_release 17 (quincy) 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:election_strategy: 1 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:2: [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] mon.c 2024-04-25T01:00:29.661 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T01:00:29.662 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T01:00:29.676 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4b43d0d1-0ff7-4061-ba99-dacfcc159ca4... 2024-04-25T01:00:29.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T01:00:29.719 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T01:00:29.756 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-04-25T01:00:29.757 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-04-25T01:00:29.758 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-04-25T01:00:29.759 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = "172.21.15.129,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on skipped message = False 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = 4b43d0d1-0ff7-4061-ba99-dacfcc159ca4 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 600 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-04-25T01:00:29.760 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug trim objects = true 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-04-25T01:00:29.761 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore block size = 96636764160 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore fsck on mount = True 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluefs = 20 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluestore = 20 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: debug rocksdb = 10 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd full ratio = 0.9 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd nearfull ratio = 0.8 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: osd failsafe full ratio = 0.95 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = bluestore 2024-04-25T01:00:29.762 INFO:teuthology.orchestra.run.smithi129.stdout: osd op complaint time = 180 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: osd shutdown pgref assert = True 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug client = 20 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-04-25T01:00:29.763 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T01:00:29.764 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: mon down mkfs grace = 300 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: mon op complaint time = 120 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-04-25T01:00:29.765 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout: client mount timeout = 600 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout: debug client = 20 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout: rados mon op timeout = 900 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout: rados osd op timeout = 900 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.b] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.c] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d] 2024-04-25T01:00:29.766 INFO:teuthology.orchestra.run.smithi129.stdout:[mds] 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: debug mds = 20 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: debug mds balancer = 20 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: mds debug frag = True 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: mds debug scatterstat = True 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: mds op complaint time = 180 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: mds verify scatter = True 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: osd op complaint time = 180 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: rados mon op timeout = 900 2024-04-25T01:00:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: rados osd op timeout = 900 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-04-25T01:00:29.768 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.769 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.770 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.129,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on skipped message = False 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 4b43d0d1-0ff7-4061-ba99-dacfcc159ca4 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-04-25T01:00:29.771 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug trim objects = true 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.772 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-04-25T01:00:29.773 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 20 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 20 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 10 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: osd shutdown pgref assert = True 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.774 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-04-25T01:00:29.775 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T01:00:29.776 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: mon down mkfs grace = 300 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-04-25T01:00:29.777 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-04-25T01:00:29.778 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-04-25T01:00:29.779 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-04-25T01:00:29.779 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-04-25T01:00:29.779 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-04-25T01:00:29.779 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-04-25T01:00:29.779 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-04-25T01:00:29.779 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-04-25T01:00:29.780 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T01:00:29.780 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T01:00:29.874 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T01:00:29.874 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:29.874 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T01:00:29.889 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:29.889 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T01:00:29.944 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:00:29.944 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:29.944 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T01:00:29.944 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:00:30.017 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:00:30.017 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:30.072 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:00:30.072 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:30.072 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T01:00:30.072 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:00:30.105 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:00:30.105 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:30.160 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T01:00:30.160 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T01:00:30.161 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T01:00:30.208 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T01:00:30.210 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T01:00:30.268 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T01:00:30.270 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T01:00:30.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T01:00:30.318 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T01:00:30.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T01:00:30.344 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T01:00:30.432 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T01:00:30.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T01:00:30.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T01:00:30.506 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T01:00:30.508 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T01:00:30.532 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T01:00:30.619 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T01:00:30.621 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T01:00:30.645 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T01:00:30.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T01:00:30.684 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T01:00:30.696 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T01:00:30.734 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T01:00:30.745 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T01:00:30.746 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T01:00:30.746 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T01:00:30.769 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:30.769 INFO:tasks.ceph:role: osd.0 2024-04-25T01:00:30.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:00:30.769 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T01:00:30.839 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:30.840 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:31.351 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T01:00:31.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T01:00:31.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T01:00:31.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T01:00:31.536 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T01:00:31.600 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:31.600 INFO:tasks.ceph:role: osd.1 2024-04-25T01:00:31.600 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:00:31.600 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T01:00:31.671 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:31.724 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:31.725 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:32.118 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T01:00:32.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T01:00:32.120 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T01:00:32.212 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T01:00:32.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T01:00:32.304 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:32.304 INFO:tasks.ceph:role: osd.2 2024-04-25T01:00:32.304 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:00:32.304 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T01:00:32.375 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:32.376 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:32.376 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:32.376 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:32.376 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:32.861 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T01:00:32.863 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T01:00:32.863 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T01:00:32.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T01:00:33.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T01:00:33.087 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:33.087 INFO:tasks.ceph:role: osd.3 2024-04-25T01:00:33.088 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:00:33.088 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:33.159 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:33.160 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:33.659 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-04-25T01:00:33.661 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-04-25T01:00:33.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T01:00:33.779 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T01:00:33.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:33.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:33.897+0000 7f3c9221b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T01:00:33.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:33.897+0000 7f3c9221b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T01:00:33.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:33.897+0000 7f3c9221b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:00:33.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:33.897+0000 7f3c9221b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:00:33.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:33.897+0000 7f3c9221b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:00:33.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:33.898+0000 7f3c9221b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T01:00:36.750 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:00:36.778 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:36.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:36.866+0000 7f22a4eaf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T01:00:36.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:36.866+0000 7f22a4eaf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T01:00:36.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:36.866+0000 7f22a4eaf640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:00:36.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:36.866+0000 7f22a4eaf640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:00:36.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:36.866+0000 7f22a4eaf640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:00:36.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:36.867+0000 7f22a4eaf640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T01:00:39.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:00:39.741 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:39.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:39.829+0000 7f1eb9a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T01:00:39.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:39.830+0000 7f1eb9a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T01:00:39.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:39.830+0000 7f1eb9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:00:39.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:39.830+0000 7f1eb9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:00:39.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:39.830+0000 7f1eb9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:00:39.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:39.831+0000 7f1eb9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T01:00:42.696 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:00:42.725 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:42.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:42.812+0000 7fc71e544640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T01:00:42.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:42.812+0000 7fc71e544640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T01:00:42.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:42.812+0000 7fc71e544640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:00:42.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:42.812+0000 7fc71e544640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:00:42.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:42.812+0000 7fc71e544640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:00:42.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:00:42.813+0000 7fc71e544640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T01:00:45.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:00:45.680 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T01:00:45.709 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:45.710 INFO:tasks.ceph:role: osd.4 2024-04-25T01:00:45.710 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:00:45.710 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T01:00:45.784 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:45.784 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:45.784 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:45.785 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:45.785 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:45.785 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:45.785 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:46.305 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-04-25T01:00:46.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-04-25T01:00:46.308 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T01:00:46.448 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T01:00:46.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T01:00:46.550 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:46.550 INFO:tasks.ceph:role: osd.5 2024-04-25T01:00:46.550 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:00:46.551 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T01:00:46.622 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:46.623 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:47.069 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-04-25T01:00:47.071 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-04-25T01:00:47.072 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T01:00:47.154 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T01:00:47.227 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T01:00:47.293 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:47.293 INFO:tasks.ceph:role: osd.6 2024-04-25T01:00:47.293 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:00:47.293 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:47.366 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:47.367 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:47.853 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-04-25T01:00:47.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-04-25T01:00:47.855 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T01:00:47.967 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T01:00:47.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T01:00:48.066 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:00:48.066 INFO:tasks.ceph:role: osd.7 2024-04-25T01:00:48.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:00:48.067 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T01:00:48.138 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:00:48.138 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:00:48.138 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:00:48.139 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:00:48.645 INFO:teuthology.orchestra.run.smithi171.stdout:Discarding blocks...Done. 2024-04-25T01:00:48.647 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-04-25T01:00:48.647 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T01:00:48.785 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T01:00:48.817 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:48.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:48.924+0000 7f2c974e3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T01:00:48.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:48.925+0000 7f2c974e3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T01:00:48.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:48.925+0000 7f2c974e3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:00:48.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:48.925+0000 7f2c974e3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:00:48.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:48.925+0000 7f2c974e3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:00:48.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:48.925+0000 7f2c974e3640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T01:00:51.780 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:00:51.808 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:51.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:51.897+0000 7f0eb5422640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T01:00:51.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:51.897+0000 7f0eb5422640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T01:00:51.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:51.897+0000 7f0eb5422640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:00:51.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:51.897+0000 7f0eb5422640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:00:51.899 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:51.897+0000 7f0eb5422640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:00:51.899 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:51.898+0000 7f0eb5422640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T01:00:54.745 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:00:54.773 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:54.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:54.861+0000 7fa4c10e5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T01:00:54.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:54.861+0000 7fa4c10e5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T01:00:54.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:54.861+0000 7fa4c10e5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:00:54.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:54.861+0000 7fa4c10e5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:00:54.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:54.861+0000 7fa4c10e5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:00:54.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:54.861+0000 7fa4c10e5640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T01:00:57.715 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T01:00:57.743 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:00:57.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:00:57.795 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:00:57.832 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:57.832+0000 7f8230a2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T01:00:57.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:57.832+0000 7f8230a2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T01:00:57.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:57.832+0000 7f8230a2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:00:57.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:57.832+0000 7f8230a2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:00:57.833 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:57.832+0000 7f8230a2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:00:57.834 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:00:57.833+0000 7f8230a2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T01:01:00.684 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T01:01:00.710 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T01:01:00.710 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:00.710 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T01:01:00.739 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:00.739 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T01:01:00.805 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:00.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T01:01:00.870 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:00.870 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T01:01:00.935 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:00.935 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T01:01:00.999 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:00.999 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T01:01:01.063 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:01.063 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T01:01:01.127 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.127 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T01:01:01.154 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.154 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T01:01:01.221 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.221 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T01:01:01.288 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.288 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T01:01:01.352 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.352 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T01:01:01.417 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.417 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T01:01:01.482 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.482 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T01:01:01.546 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:01.546 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T01:01:01.561 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:01.561 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T01:01:01.601 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T01:01:01.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T01:01:01.604 DEBUG:teuthology.orchestra.run.smithi171:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.y] 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylm3phzDBAAh9Hnao23vZXCoxCJdeTlsw== 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a] 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylmHusIExAAukgTPHD/PaiMJiP7JYP8pw== 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c] 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylmr87MGRAAu7XSB/hyASOaZWFJAKecgQ== 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAxqylmkkCONRAASg7tac6cJCGvHyFoWzXumw== 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQA0qylmlSmuMxAAcsN+GHdIhzwgjw04wCvXVQ== 2024-04-25T01:01:01.645 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQA3qylm4RGFMRAAfIquK3LKJOMUNvb1rSI4VA== 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQA6qylmYbt7MBAAr9Br77LqYNaVKLCtO3kLcw== 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylmpG4HEBAAuGMF+H8SE4RgEmIzwbSrRw== 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylmdNs0HhAAfovaEMbD8M4JRp9OACyFWw== 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylm0DrzJBAATf9n72VgYcjObu1gWZ30nw== 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBAqylmI3kuNxAAEsveh6r7Fk0iUk1CS2GoSw== 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2024-04-25T01:01:01.646 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBDqylmM46NNRAAraR68p+sN7+6mhnHj41mxQ== 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.6] 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBGqylmx7RiMxAA2j9P/BfquzFxxGHtcZwnHw== 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.7] 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBJqylm6PGrMRAAosKGej/AJjVVOQrznc66qA== 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylm5vzVKBAA5v9x8BTq5UjJaTD/Hki+tg== 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-04-25T01:01:01.647 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQAuqylmgR/NKxAAqDs81peCjgFrC+ru6O7gmg== 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.y] 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylm3phzDBAAh9Hnao23vZXCoxCJdeTlsw== 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylmHusIExAAukgTPHD/PaiMJiP7JYP8pw== 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylmr87MGRAAu7XSB/hyASOaZWFJAKecgQ== 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.0] 2024-04-25T01:01:01.663 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAxqylmkkCONRAASg7tac6cJCGvHyFoWzXumw== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.1] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQA0qylmlSmuMxAAcsN+GHdIhzwgjw04wCvXVQ== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.2] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQA3qylm4RGFMRAAfIquK3LKJOMUNvb1rSI4VA== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.3] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQA6qylmYbt7MBAAr9Br77LqYNaVKLCtO3kLcw== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.x] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylmpG4HEBAAuGMF+H8SE4RgEmIzwbSrRw== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylmdNs0HhAAfovaEMbD8M4JRp9OACyFWw== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylm0DrzJBAATf9n72VgYcjObu1gWZ30nw== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.4] 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQBAqylmI3kuNxAAEsveh6r7Fk0iUk1CS2GoSw== 2024-04-25T01:01:01.664 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.5] 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQBDqylmM46NNRAAraR68p+sN7+6mhnHj41mxQ== 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.6] 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQBGqylmx7RiMxAA2j9P/BfquzFxxGHtcZwnHw== 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.7] 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQBJqylm6PGrMRAAosKGej/AJjVVOQrznc66qA== 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout:[client.0] 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylm5vzVKBAA5v9x8BTq5UjJaTD/Hki+tg== 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout:[client.1] 2024-04-25T01:01:01.665 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQAuqylmgR/NKxAAqDs81peCjgFrC+ru6O7gmg== 2024-04-25T01:01:01.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:01:01.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:01:01.755 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:01.757 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:01.810 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:01.812 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:01.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:01.865 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:01.913 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:01.915 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:01.963 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:01.966 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.014 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.016 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:01:02.066 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:01:02.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:02.116 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:02.167 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:02.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:01:02.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.224 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.327 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.376 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.496 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:01:02.547 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:01:02.549 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:01:02.602 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:01:02.604 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:01:02.657 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T01:01:02.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T01:01:02.680 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:01:02.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T01:01:02.796 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T01:01:02.820 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:01:02.913 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T01:01:02.936 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T01:01:02.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:01:03.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T01:01:03.116 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:01:03.119 DEBUG:teuthology.orchestra.run.smithi171:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:01:03.173 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T01:01:03.174 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:01:03.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mon -f --cluster ceph -i a 2024-04-25T01:01:03.177 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:01:03.177 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T01:01:03.177 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mon -f --cluster ceph -i b 2024-04-25T01:01:03.214 INFO:tasks.ceph.mon.b:Started 2024-04-25T01:01:03.214 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:01:03.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mon -f --cluster ceph -i c 2024-04-25T01:01:03.216 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:01:03.216 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T01:01:03.216 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T01:01:03.216 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-04-25T01:01:03.218 INFO:tasks.ceph.mgr.y:Started 2024-04-25T01:01:03.219 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T01:01:03.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-04-25T01:01:03.220 INFO:tasks.ceph.mgr.x:Started 2024-04-25T01:01:03.221 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T01:01:03.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T01:01:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.318+0000 7f56744a5640 1 Processor -- start 2024-04-25T01:01:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.319+0000 7f56744a5640 1 -- start start 2024-04-25T01:01:03.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.319+0000 7f56744a5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:03.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.319+0000 7f56744a5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:03.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.319+0000 7f56744a5640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c153280 con 0x7f566c0c7930 2024-04-25T01:01:03.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.319+0000 7f56744a5640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f566c153b90 con 0x7f566c156870 2024-04-25T01:01:03.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.319+0000 7f56744a5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f566c0a72d0 con 0x7f566c1507b0 2024-04-25T01:01:03.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:03.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59894/0 (socket says 172.21.15.129:59894) 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 learned_addr learned my addr 172.21.15.129:0/1205010644 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:01:03.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:03.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:03.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:03.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.320+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:01:03.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:03.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:01:03.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:03.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:03.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:03.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:01:03.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:03.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:03.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:03.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.521+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:01:03.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:03.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:03.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:03.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:03.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:03.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:01:03.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:01:03.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:03.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:03.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:03.922+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:04.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:04.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:04.723+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 msgr2=0x7f566c0a6cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.171:3300/0 conn(0x7f566c1507b0 0x7f566c0a6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 -- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 msgr2=0x7f566c156d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 --2- 172.21.15.129:0/1205010644 >> v2:172.21.15.129:3300/0 conn(0x7f566c156870 0x7f566c156d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 -- 172.21.15.129:0/1205010644 >> v1:172.21.15.171:6789/0 conn(0x7f566c0c7930 legacy=0x7f566c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 -- 172.21.15.129:0/1205010644 --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c153540 con 0x7f5650006170 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5663fff640 1 -- 172.21.15.129:0/1205010644 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f566c153b90 con 0x7f5650003690 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:06.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:06.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:06.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.320+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:01:06.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.521+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:06.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.521+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:06.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.521+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:06.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.521+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:06.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.521+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:01:06.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.922+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:06.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.922+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:06.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.922+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:06.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.922+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:06.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:06.922+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:01:07.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:07.723+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:07.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:07.723+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T01:01:07.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:07.723+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:01:07.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:07.723+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:01:07.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:07.723+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.489+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3039604104 0 0) 0x7f566c153540 con 0x7f5650006170 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.489+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5658003520 con 0x7f5650006170 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 802546733 0 0) 0x7f5658003520 con 0x7f5650006170 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 msgr2=0x7f5650005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5650003690 0x7f5650005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565000afb0 con 0x7f5650006170 2024-04-25T01:01:08.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f565c0016f0 con 0x7f5650006170 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 >> v1:172.21.15.129:6789/0 conn(0x7f5650006170 legacy=0x7f5650008610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672a42640 1 -- 172.21.15.129:0/1205010644 reap_dead start 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f5658003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658004800 0x7f5658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658005290 0x7f565800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f566c153b90 con 0x7f56580039b0 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f565800c1e0 con 0x7f5658004800 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f565800c3e0 con 0x7f5658005290 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f5658003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672a42640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658005290 0x7f565800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:08.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5671a40640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658004800 0x7f5658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:08.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658005290 msgr2=0x7f565800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:08.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658005290 0x7f565800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658004800 msgr2=0x7f5658004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658004800 0x7f5658004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.490+0000 7f5672241640 1 -- 172.21.15.129:0/1205010644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5658003520 con 0x7f56580039b0 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.491+0000 7f5672241640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f5658003db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5668019cc0 tx=0x7f5658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286cdeb5e5702f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.491+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668032070 con 0x7f56580039b0 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.491+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566802cd00 con 0x7f56580039b0 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.491+0000 7f567123f640 1 -- 172.21.15.129:0/1205010644 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668019940 con 0x7f56580039b0 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 -- 172.21.15.129:0/1205010644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 msgr2=0x7f5658003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f5658003db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5668019cc0 tx=0x7f5658005830 comp rx=0 tx=0).stop 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 -- 172.21.15.129:0/1205010644 shutdown_connections 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658005290 0x7f565800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658004800 0x7f5658004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 --2- 172.21.15.129:0/1205010644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f5658003db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 -- 172.21.15.129:0/1205010644 >> 172.21.15.129:0/1205010644 conn(0x7f566c0a9730 msgr2=0x7f566c0a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:08.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 -- 172.21.15.129:0/1205010644 shutdown_connections 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.492+0000 7f56744a5640 1 -- 172.21.15.129:0/1205010644 wait complete. 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.494+0000 7f56744a5640 1 Processor -- start 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.494+0000 7f56744a5640 1 -- start start 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f56744a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f566c0c1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f56744a5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658004800 0x7f566c1ee270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f56744a5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658005290 0x7f566c1ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f56744a5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f566c150b90 con 0x7f56580039b0 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f56744a5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f566c0aa580 con 0x7f5658005290 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f56744a5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f566c1edd20 con 0x7f5658004800 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f566c0c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f566c0c1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57668/0 (socket says 172.21.15.129:57668) 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 -- 172.21.15.129:0/3524554352 learned_addr learned my addr 172.21.15.129:0/3524554352 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5671a40640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658004800 0x7f566c1ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:08.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672a42640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658005290 0x7f566c1ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 -- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658004800 msgr2=0x7f566c1ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658004800 0x7f566c1ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 -- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658005290 msgr2=0x7f566c1ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658005290 0x7f566c1ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 -- 172.21.15.129:0/3524554352 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c154f40 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5672241640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f566c0c1020 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5668038040 tx=0x7f56680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.495+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668044020 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.496+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5668003610 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.496+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c156070 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.496+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668019940 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.496+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c0ac580 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.496+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5668047020 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.496+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f566803f890 con 0x7f56580039b0 2024-04-25T01:01:08.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.497+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c0ac3a0 con 0x7f56580039b0 2024-04-25T01:01:08.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.499+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f566c0ac3a0 con 0x7f56580039b0 2024-04-25T01:01:08.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.564+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f566c075550 con 0x7f56580039b0 2024-04-25T01:01:08.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f5662ffd640 1 -- 172.21.15.129:0/3524554352 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f566804e030 con 0x7f56580039b0 2024-04-25T01:01:08.566 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 msgr2=0x7f566c0c1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f566c0c1020 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5668038040 tx=0x7f56680046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 shutdown_connections 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5658004800 0x7f566c1ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5658005290 0x7f566c1ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 --2- 172.21.15.129:0/3524554352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56580039b0 0x7f566c0c1020 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:08.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.566+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 >> 172.21.15.129:0/3524554352 conn(0x7f566c0a9730 msgr2=0x7f566c0a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:08.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.567+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 shutdown_connections 2024-04-25T01:01:08.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:08.567+0000 7f56744a5640 1 -- 172.21.15.129:0/3524554352 wait complete. 2024-04-25T01:01:08.578 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T01:01:08.578 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T01:01:08.578 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:08.579 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T01:01:08.602 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:08.603 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T01:01:08.613 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:08.613+0000 7fb9a7523040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:01:08.666 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:08.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T01:01:08.729 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:08.729 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T01:01:08.772 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:08.772+0000 7fb9a7523040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:01:08.792 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:08.792 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T01:01:08.816 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:08.816 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T01:01:08.827 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:08.827+0000 7fb9a7523040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:01:08.850 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:08.850 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T01:01:08.913 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:01:08.913 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T01:01:08.977 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new d671da37-185a-43f0-b212-f63d250d2c5a 0 2024-04-25T01:01:09.083 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:09.082+0000 7fb9a7523040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:01:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.114+0000 7f80d9488640 1 Processor -- start 2024-04-25T01:01:09.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d9488640 1 -- start start 2024-04-25T01:01:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d9488640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d406c990 0x7f80d406ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d9488640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d40c5380 con 0x7f80d4158330 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d9488640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80d40c5660 con 0x7f80d4159440 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d9488640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f80d40c5f90 con 0x7f80d406c990 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d406c990 0x7f80d406ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d27fc640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f80d4159440 0x7f80d40c7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:60438/0 (socket says 172.21.15.171:60438) 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.115+0000 7f80d27fc640 1 -- 172.21.15.171:0/848121611 learned_addr learned my addr 172.21.15.171:0/848121611 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.116+0000 7f80d37fe640 1 -- 172.21.15.171:0/848121611 >> v1:172.21.15.171:6789/0 conn(0x7f80d4159440 legacy=0x7f80d40c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.116+0000 7f80d37fe640 1 -- 172.21.15.171:0/848121611 >> v1:172.21.15.129:6789/0 conn(0x7f80d4158330 legacy=0x7f80d40c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.116+0000 7f80d1ffb640 1 -- 172.21.15.171:0/848121611 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122844262 0 0) 0x7f80d40c5660 con 0x7f80d4159440 2024-04-25T01:01:09.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.116+0000 7f80d37fe640 1 -- 172.21.15.171:0/848121611 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d40c5940 con 0x7f80d406c990 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.116+0000 7f80d37fe640 1 --2- 172.21.15.171:0/848121611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d406c990 0x7f80d406ce20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009dc0 tx=0x7f80cc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8327627110c499bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.116+0000 7f80d1ffb640 1 -- 172.21.15.171:0/848121611 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007740 con 0x7f80d406c990 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d1ffb640 1 -- 172.21.15.171:0/848121611 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc0078e0 con 0x7f80d406c990 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d1ffb640 1 -- 172.21.15.171:0/848121611 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007cb0 con 0x7f80d406c990 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 -- 172.21.15.171:0/848121611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d406c990 msgr2=0x7f80d406ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 --2- 172.21.15.171:0/848121611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d406c990 0x7f80d406ce20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009dc0 tx=0x7f80cc02a5f0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 -- 172.21.15.171:0/848121611 shutdown_connections 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 --2- 172.21.15.171:0/848121611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d406c990 0x7f80d406ce20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 -- 172.21.15.171:0/848121611 >> 172.21.15.171:0/848121611 conn(0x7f80d40c8a90 msgr2=0x7f80d40c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 -- 172.21.15.171:0/848121611 shutdown_connections 2024-04-25T01:01:09.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.117+0000 7f80d9488640 1 -- 172.21.15.171:0/848121611 wait complete. 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.118+0000 7f80d9488640 1 Processor -- start 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 -- start start 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 0x7f80d40c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d4158330 0x7f80d4152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d4159440 0x7f80d41ec530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4158710 con 0x7f80d4158330 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41598c0 con 0x7f80d406c990 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d9488640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f80d406d360 con 0x7f80d4159440 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 0x7f80d40c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 0x7f80d40c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53498/0 (socket says 172.21.15.171:53498) 2024-04-25T01:01:09.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d2ffd640 1 -- 172.21.15.171:0/3878668585 learned_addr learned my addr 172.21.15.171:0/3878668585 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.119+0000 7f80d37fe640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d4159440 0x7f80d41ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d27fc640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d4158330 0x7f80d4152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d2ffd640 1 -- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d4159440 msgr2=0x7f80d41ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d2ffd640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d4159440 0x7f80d41ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d2ffd640 1 -- 172.21.15.171:0/3878668585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d4158330 msgr2=0x7f80d4152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d2ffd640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d4158330 0x7f80d4152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d2ffd640 1 -- 172.21.15.171:0/3878668585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d41df4f0 con 0x7f80d406c990 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d27fc640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d4158330 0x7f80d4152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d2ffd640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 0x7f80d40c4380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c410 tx=0x7f80c400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017070 con 0x7f80d406c990 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40035f0 con 0x7f80d406c990 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d41df7d0 con 0x7f80d406c990 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4003a20 con 0x7f80d406c990 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41dfd90 con 0x7f80d406c990 2024-04-25T01:01:09.123 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80c401a070 con 0x7f80d406c990 2024-04-25T01:01:09.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.120+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f80c4012740 con 0x7f80d406c990 2024-04-25T01:01:09.124 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.121+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d4075550 con 0x7f80d406c990 2024-04-25T01:01:09.125 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.125+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f80c4021020 con 0x7f80d406c990 2024-04-25T01:01:09.132 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:09.132+0000 7fb9a7523040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:01:09.198 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.197+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d671da37-185a-43f0-b212-f63d250d2c5a", "id": 0} v 0) -- 0x7f80d40c6d80 con 0x7f80d406c990 2024-04-25T01:01:09.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.201+0000 7f80cb7fe640 1 -- 172.21.15.171:0/3878668585 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d671da37-185a-43f0-b212-f63d250d2c5a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f80c4003d50 con 0x7f80d406c990 2024-04-25T01:01:09.201 INFO:teuthology.orchestra.run.smithi171.stdout:0 2024-04-25T01:01:09.202 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.201+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 msgr2=0x7f80d40c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.202 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.201+0000 7f80d9488640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 0x7f80d40c4380 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c410 tx=0x7f80c400c8d0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.201+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 shutdown_connections 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.201+0000 7f80d9488640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80d4159440 0x7f80d41ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.202+0000 7f80d9488640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d4158330 0x7f80d4152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.202+0000 7f80d9488640 1 --2- 172.21.15.171:0/3878668585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80d406c990 0x7f80d40c4380 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.202+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 >> 172.21.15.171:0/3878668585 conn(0x7f80d40c8a90 msgr2=0x7f80d4156c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.202+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 shutdown_connections 2024-04-25T01:01:09.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.202+0000 7f80d9488640 1 -- 172.21.15.171:0/3878668585 wait complete. 2024-04-25T01:01:09.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new c03a61c0-f685-4a43-acec-4edbe0dfd835 1 2024-04-25T01:01:09.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.310+0000 7fd566b53640 1 Processor -- start 2024-04-25T01:01:09.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.310+0000 7fd566b53640 1 -- start start 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd566b53640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd56006fc20 0x7fd5600c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd566b53640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd560153030 con 0x7fd5600c4960 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd566b53640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5601524b0 con 0x7fd5600c6210 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd566b53640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fd5600c7300 con 0x7fd56006fc20 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd55ffff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd56006fc20 0x7fd5600c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd5650f0640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fd5600c6210 0x7fd56015c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:60452/0 (socket says 172.21.15.171:60452) 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd55ffff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd56006fc20 0x7fd5600c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53510/0 (socket says 172.21.15.171:53510) 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd5650f0640 1 -- 172.21.15.171:0/2400045156 learned_addr learned my addr 172.21.15.171:0/2400045156 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-04-25T01:01:09.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd55ffff640 1 -- 172.21.15.171:0/2400045156 >> v1:172.21.15.171:6789/0 conn(0x7fd5600c6210 legacy=0x7fd56015c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd55ffff640 1 -- 172.21.15.171:0/2400045156 >> v1:172.21.15.129:6789/0 conn(0x7fd5600c4960 legacy=0x7fd560071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:09.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.311+0000 7fd55ffff640 1 -- 172.21.15.171:0/2400045156 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd560152790 con 0x7fd56006fc20 2024-04-25T01:01:09.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.312+0000 7fd55ffff640 1 --2- 172.21.15.171:0/2400045156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5600c5cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0097a0 tx=0x7fd54c02a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21812778c688b7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.312+0000 7fd55f7fe640 1 -- 172.21.15.171:0/2400045156 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c037070 con 0x7fd56006fc20 2024-04-25T01:01:09.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.312+0000 7fd55f7fe640 1 -- 172.21.15.171:0/2400045156 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c02ae50 con 0x7fd56006fc20 2024-04-25T01:01:09.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.312+0000 7fd55f7fe640 1 -- 172.21.15.171:0/2400045156 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c032660 con 0x7fd56006fc20 2024-04-25T01:01:09.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 -- 172.21.15.171:0/2400045156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 msgr2=0x7fd5600c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.317 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 --2- 172.21.15.171:0/2400045156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5600c5cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0097a0 tx=0x7fd54c02a140 comp rx=0 tx=0).stop 2024-04-25T01:01:09.317 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 -- 172.21.15.171:0/2400045156 shutdown_connections 2024-04-25T01:01:09.317 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 --2- 172.21.15.171:0/2400045156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5600c5cd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.317 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 -- 172.21.15.171:0/2400045156 >> 172.21.15.171:0/2400045156 conn(0x7fd5600c7a80 msgr2=0x7fd5600c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 -- 172.21.15.171:0/2400045156 shutdown_connections 2024-04-25T01:01:09.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.313+0000 7fd566b53640 1 -- 172.21.15.171:0/2400045156 wait complete. 2024-04-25T01:01:09.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.314+0000 7fd566b53640 1 Processor -- start 2024-04-25T01:01:09.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.314+0000 7fd566b53640 1 -- start start 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd566b53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5601eea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd566b53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5600c4960 0x7fd5601e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd566b53640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 0x7fd5601e1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd566b53640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd5600c4d40 con 0x7fd5600c4960 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd566b53640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd560152d50 con 0x7fd56006fc20 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd566b53640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd5600c7710 con 0x7fd5600c6210 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5648ef640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5601eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 0x7fd5601e1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 0x7fd5601e1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40250/0 (socket says 172.21.15.171:40250) 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 -- 172.21.15.171:0/2250410294 learned_addr learned my addr 172.21.15.171:0/2250410294 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:09.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd55ffff640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5600c4960 0x7fd5601e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 -- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 msgr2=0x7fd5601eea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5601eea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 -- 172.21.15.171:0/2250410294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5600c4960 msgr2=0x7fd5601e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5600c4960 0x7fd5601e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 -- 172.21.15.171:0/2250410294 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5601ec820 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5648ef640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5601eea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.315+0000 7fd5650f0640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 0x7fd5601e1010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd55800ca80 tx=0x7fd55800cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558017070 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd55800ac30 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5601ecb00 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558012400 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601ed0c0 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd55801a070 con 0x7fd5600c6210 2024-04-25T01:01:09.322 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.316+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd55801c830 con 0x7fd5600c6210 2024-04-25T01:01:09.323 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.317+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5601ed2c0 con 0x7fd5600c6210 2024-04-25T01:01:09.323 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.319+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd5601ed2c0 con 0x7fd5600c6210 2024-04-25T01:01:09.392 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.391+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c03a61c0-f685-4a43-acec-4edbe0dfd835", "id": 1} v 0) -- 0x7fd560075550 con 0x7fd5600c6210 2024-04-25T01:01:09.395 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.394+0000 7fd55d7fa640 1 -- 172.21.15.171:0/2250410294 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c03a61c0-f685-4a43-acec-4edbe0dfd835", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd55801cb50 con 0x7fd5600c6210 2024-04-25T01:01:09.395 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-04-25T01:01:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 msgr2=0x7fd5601e1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 0x7fd5601e1010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd55800ca80 tx=0x7fd55800cf40 comp rx=0 tx=0).stop 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 shutdown_connections 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd5600c6210 0x7fd5601e1010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5600c4960 0x7fd5601e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 --2- 172.21.15.171:0/2250410294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd56006fc20 0x7fd5601eea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 >> 172.21.15.171:0/2250410294 conn(0x7fd5600c7a80 msgr2=0x7fd56014ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 shutdown_connections 2024-04-25T01:01:09.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.395+0000 7fd566b53640 1 -- 172.21.15.171:0/2250410294 wait complete. 2024-04-25T01:01:09.405 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:09.405+0000 7f4f20e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:01:09.407 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 8e4dfd32-d051-444e-8789-a462191b5440 2 2024-04-25T01:01:09.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.505+0000 7f105609e640 1 Processor -- start 2024-04-25T01:01:09.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.505+0000 7f105609e640 1 -- start start 2024-04-25T01:01:09.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f105609e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 0x7f10501591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f105609e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10501554e0 con 0x7f1050152a60 2024-04-25T01:01:09.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f105609e640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10501557a0 con 0x7f105006e7d0 2024-04-25T01:01:09.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f105609e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f10501560d0 con 0x7f105006f200 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f104ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 0x7f10501591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f104effd640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f105006e7d0 0x7f1050158ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:60462/0 (socket says 172.21.15.171:60462) 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f104ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 0x7f10501591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40252/0 (socket says 172.21.15.171:40252) 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.506+0000 7f104effd640 1 -- 172.21.15.171:0/2405536656 learned_addr learned my addr 172.21.15.171:0/2405536656 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104e7fc640 1 -- 172.21.15.171:0/2405536656 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3043293923 0 0) 0x7f10501557a0 con 0x7f105006e7d0 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104e7fc640 1 -- 172.21.15.171:0/2405536656 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1038003520 con 0x7f105006e7d0 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104ffff640 1 -- 172.21.15.171:0/2405536656 >> v1:172.21.15.171:6789/0 conn(0x7f105006e7d0 legacy=0x7f1050158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104ffff640 1 -- 172.21.15.171:0/2405536656 >> v1:172.21.15.129:6789/0 conn(0x7f1050152a60 legacy=0x7f10500c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104ffff640 1 -- 172.21.15.171:0/2405536656 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1050155a80 con 0x7f105006f200 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104ffff640 1 --2- 172.21.15.171:0/2405536656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 0x7f10501591e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1050156250 tx=0x7f104802a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c9b251988d721a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104e7fc640 1 -- 172.21.15.171:0/2405536656 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048007740 con 0x7f105006f200 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104e7fc640 1 -- 172.21.15.171:0/2405536656 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f104802ae50 con 0x7f105006f200 2024-04-25T01:01:09.509 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.507+0000 7f104e7fc640 1 -- 172.21.15.171:0/2405536656 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048007b00 con 0x7f105006f200 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 -- 172.21.15.171:0/2405536656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 msgr2=0x7f10501591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 --2- 172.21.15.171:0/2405536656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 0x7f10501591e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1050156250 tx=0x7f104802a630 comp rx=0 tx=0).stop 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 -- 172.21.15.171:0/2405536656 shutdown_connections 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 --2- 172.21.15.171:0/2405536656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f105006f200 0x7f10501591e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 -- 172.21.15.171:0/2405536656 >> 172.21.15.171:0/2405536656 conn(0x7f10500a9730 msgr2=0x7f10500a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 -- 172.21.15.171:0/2405536656 shutdown_connections 2024-04-25T01:01:09.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.508+0000 7f105609e640 1 -- 172.21.15.171:0/2405536656 wait complete. 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.509+0000 7f105609e640 1 Processor -- start 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.509+0000 7f105609e640 1 -- start start 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f105609e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 0x7f1050151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f105609e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1050152a60 0x7f10500c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f105609e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 0x7f1050146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f105609e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10500c5fb0 con 0x7f1050152a60 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f105609e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1050152e40 con 0x7f105006f200 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f105609e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f10500c6130 con 0x7f1050147860 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 0x7f1050151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 0x7f1050146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 0x7f1050146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40264/0 (socket says 172.21.15.171:40264) 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 0x7f1050151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53542/0 (socket says 172.21.15.171:53542) 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104ffff640 1 -- 172.21.15.171:0/3692824816 learned_addr learned my addr 172.21.15.171:0/3692824816 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104effd640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1050152a60 0x7f10500c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104f7fe640 1 -- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 msgr2=0x7f1050146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.510+0000 7f104f7fe640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 0x7f1050146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f104f7fe640 1 -- 172.21.15.171:0/3692824816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1050152a60 msgr2=0x7f10500c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f104f7fe640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1050152a60 0x7f10500c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f104f7fe640 1 -- 172.21.15.171:0/3692824816 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10501edf80 con 0x7f105006f200 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f104ffff640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 0x7f1050146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f104f7fe640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 0x7f1050151f60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f104400c3f0 tx=0x7f104400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440052d0 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1044005490 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10501ee1f0 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044007260 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501ee7b0 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1044007400 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.511+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f1044011590 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.512+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1050075530 con 0x7f105006f200 2024-04-25T01:01:09.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.514+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f10440035f0 con 0x7f105006f200 2024-04-25T01:01:09.559 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:09.559+0000 7f4f20e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:01:09.586 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.586+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8e4dfd32-d051-444e-8789-a462191b5440", "id": 2} v 0) -- 0x7f10500c3870 con 0x7f105006f200 2024-04-25T01:01:09.589 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.589+0000 7f1037fff640 1 -- 172.21.15.171:0/3692824816 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e4dfd32-d051-444e-8789-a462191b5440", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f1044003ac0 con 0x7f105006f200 2024-04-25T01:01:09.590 INFO:teuthology.orchestra.run.smithi171.stdout:2 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 msgr2=0x7f1050151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 0x7f1050151f60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f104400c3f0 tx=0x7f104400c8b0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 shutdown_connections 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1050147860 0x7f1050146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1050152a60 0x7f10500c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 --2- 172.21.15.171:0/3692824816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f105006f200 0x7f1050151f60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 >> 172.21.15.171:0/3692824816 conn(0x7f10500a9730 msgr2=0x7f10500c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 shutdown_connections 2024-04-25T01:01:09.591 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.590+0000 7f105609e640 1 -- 172.21.15.171:0/3692824816 wait complete. 2024-04-25T01:01:09.601 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 582dca5d-083d-4052-8146-9d3f5dce56bc 3 2024-04-25T01:01:09.612 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:09.612+0000 7f4f20e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:01:09.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.720+0000 7f9d05a12640 1 Processor -- start 2024-04-25T01:01:09.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.720+0000 7f9d05a12640 1 -- start start 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9d05a12640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9d00152d30 0x7f9d000c1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9d05a12640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d000c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9d05a12640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d00159960 con 0x7f9d000a7010 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9d05a12640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f9d000c2840 con 0x7f9d00152d30 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9d05a12640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9d00070120 con 0x7f9d001554d0 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9d00152d30 0x7f9d000c1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cff7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d000c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9d00152d30 0x7f9d000c1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53554/0 (socket says 172.21.15.171:53554) 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 -- 172.21.15.171:0/2175249186 learned_addr learned my addr 172.21.15.171:0/2175249186 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cff7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d000c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40276/0 (socket says 172.21.15.171:40276) 2024-04-25T01:01:09.724 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 -- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 msgr2=0x7f9d000c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 --2- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d000c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 -- 172.21.15.171:0/2175249186 >> v1:172.21.15.129:6789/0 conn(0x7f9d000a7010 legacy=0x7f9d0006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.721+0000 7f9cfeffd640 1 -- 172.21.15.171:0/2175249186 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d000c4280 con 0x7f9d00152d30 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.722+0000 7f9cfeffd640 1 --2- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d00152d30 0x7f9d000c1e00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009870 tx=0x7f9ce802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec98bf668e1c8bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.722+0000 7f9cfdffb640 1 -- 172.21.15.171:0/2175249186 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8037070 con 0x7f9d00152d30 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.722+0000 7f9cfdffb640 1 -- 172.21.15.171:0/2175249186 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8032440 con 0x7f9d00152d30 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.722+0000 7f9cfdffb640 1 -- 172.21.15.171:0/2175249186 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8032810 con 0x7f9d00152d30 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 -- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d00152d30 msgr2=0x7f9d000c1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 --2- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d00152d30 0x7f9d000c1e00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009870 tx=0x7f9ce802a1c0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 -- 172.21.15.171:0/2175249186 shutdown_connections 2024-04-25T01:01:09.725 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 --2- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d000c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.726 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 --2- 172.21.15.171:0/2175249186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d00152d30 0x7f9d000c1e00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.726 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 -- 172.21.15.171:0/2175249186 >> 172.21.15.171:0/2175249186 conn(0x7f9d000c7a80 msgr2=0x7f9d000c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.726 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 -- 172.21.15.171:0/2175249186 shutdown_connections 2024-04-25T01:01:09.726 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.723+0000 7f9d05a12640 1 -- 172.21.15.171:0/2175249186 wait complete. 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.724+0000 7f9d05a12640 1 Processor -- start 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.724+0000 7f9d05a12640 1 -- start start 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9d05a12640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 0x7f9d000a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9d05a12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00152d30 0x7f9d001ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9d05a12640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d00158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9d05a12640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d000c2480 con 0x7f9d00152d30 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9d05a12640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9d00159580 con 0x7f9d000a7010 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9d05a12640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9d000c2600 con 0x7f9d001554d0 2024-04-25T01:01:09.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 0x7f9d000a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cff7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d00158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 0x7f9d000a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53564/0 (socket says 172.21.15.171:53564) 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 -- 172.21.15.171:0/1091741664 learned_addr learned my addr 172.21.15.171:0/1091741664 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cff7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d00158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40292/0 (socket says 172.21.15.171:40292) 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfe7fc640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00152d30 0x7f9d001ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 -- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 msgr2=0x7f9d00158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d00158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 -- 172.21.15.171:0/1091741664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00152d30 msgr2=0x7f9d001ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00152d30 0x7f9d001ebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cfeffd640 1 -- 172.21.15.171:0/1091741664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0006ec80 con 0x7f9d000a7010 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.725+0000 7f9cff7fe640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d00158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9cfeffd640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 0x7f9d000a7de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8000c00 tx=0x7f9ce80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8037040 con 0x7f9d000a7010 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8042070 con 0x7f9d000a7010 2024-04-25T01:01:09.733 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d000a8320 con 0x7f9d000a7010 2024-04-25T01:01:09.734 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8030880 con 0x7f9d000a7010 2024-04-25T01:01:09.734 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d001ef3b0 con 0x7f9d000a7010 2024-04-25T01:01:09.734 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ce802ad10 con 0x7f9d000a7010 2024-04-25T01:01:09.734 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.726+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f9ce8047d00 con 0x7f9d000a7010 2024-04-25T01:01:09.734 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.727+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d00075550 con 0x7f9d000a7010 2024-04-25T01:01:09.734 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.729+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9ce8047460 con 0x7f9d000a7010 2024-04-25T01:01:09.800 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.800+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd new", "uuid": "582dca5d-083d-4052-8146-9d3f5dce56bc", "id": 3} v 0) -- 0x7f9d00156410 con 0x7f9d000a7010 2024-04-25T01:01:09.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.805+0000 7f9cf7fff640 1 -- 172.21.15.171:0/1091741664 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "582dca5d-083d-4052-8146-9d3f5dce56bc", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f9ce8047870 con 0x7f9d000a7010 2024-04-25T01:01:09.806 INFO:teuthology.orchestra.run.smithi171.stdout:3 2024-04-25T01:01:09.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 msgr2=0x7f9d000a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 0x7f9d000a7de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8000c00 tx=0x7f9ce80046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 shutdown_connections 2024-04-25T01:01:09.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d001554d0 0x7f9d00158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d00152d30 0x7f9d001ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 --2- 172.21.15.171:0/1091741664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d000a7010 0x7f9d000a7de0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 >> 172.21.15.171:0/1091741664 conn(0x7f9d000c7a80 msgr2=0x7f9d00150d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 shutdown_connections 2024-04-25T01:01:09.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.806+0000 7f9d05a12640 1 -- 172.21.15.171:0/1091741664 wait complete. 2024-04-25T01:01:09.818 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 5bfc9318-9d58-456c-81de-d3e7d2231fb1 4 2024-04-25T01:01:09.863 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:09.862+0000 7f4f20e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:01:09.876 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:09.876+0000 7fb9a7523040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:01:09.908 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:09.907+0000 7f4f20e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:01:09.916 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.915+0000 7fb2adbce640 1 Processor -- start 2024-04-25T01:01:09.916 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.915+0000 7fb2adbce640 1 -- start start 2024-04-25T01:01:09.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2adbce640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb2a80c4960 0x7fb2a8071790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2adbce640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb2a806fc20 0x7fb2a80c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2adbce640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a8153030 con 0x7fb2a8071cd0 2024-04-25T01:01:09.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2adbce640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a80c7300 con 0x7fb2a80c4960 2024-04-25T01:01:09.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2adbce640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a81524b0 con 0x7fb2a806fc20 2024-04-25T01:01:09.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a7fff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb2a806fc20 0x7fb2a80c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb29ffff640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fb2a8071cd0 0x7fb2a80c5cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:60474/0 (socket says 172.21.15.171:60474) 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb29ffff640 1 -- 172.21.15.171:0/7880663 learned_addr learned my addr 172.21.15.171:0/7880663 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a77fe640 1 --2- 172.21.15.171:0/7880663 >> v2:172.21.15.129:3300/0 conn(0x7fb2a80c4960 0x7fb2a8071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a7fff640 1 -- 172.21.15.171:0/7880663 >> v2:172.21.15.129:3300/0 conn(0x7fb2a80c4960 msgr2=0x7fb2a8071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a7fff640 1 --2- 172.21.15.171:0/7880663 >> v2:172.21.15.129:3300/0 conn(0x7fb2a80c4960 0x7fb2a8071790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a7fff640 1 -- 172.21.15.171:0/7880663 >> v1:172.21.15.171:6789/0 conn(0x7fb2a8071cd0 legacy=0x7fb2a80c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a6ffd640 1 -- 172.21.15.171:0/7880663 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2889324686 0 0) 0x7fb2a8153030 con 0x7fb2a8071cd0 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.916+0000 7fb2a7fff640 1 -- 172.21.15.171:0/7880663 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a8152790 con 0x7fb2a806fc20 2024-04-25T01:01:09.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.917+0000 7fb2a7fff640 1 --2- 172.21.15.171:0/7880663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a80c63e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80c2f40 tx=0x7fb2a002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c1bb2076c2745c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2a6ffd640 1 -- 172.21.15.171:0/7880663 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007740 con 0x7fb2a806fc20 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2a6ffd640 1 -- 172.21.15.171:0/7880663 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a00078e0 con 0x7fb2a806fc20 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2a6ffd640 1 -- 172.21.15.171:0/7880663 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007cb0 con 0x7fb2a806fc20 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 -- 172.21.15.171:0/7880663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 msgr2=0x7fb2a80c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 --2- 172.21.15.171:0/7880663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a80c63e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80c2f40 tx=0x7fb2a002a5f0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 -- 172.21.15.171:0/7880663 shutdown_connections 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 --2- 172.21.15.171:0/7880663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a80c63e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 --2- 172.21.15.171:0/7880663 >> v2:172.21.15.129:3300/0 conn(0x7fb2a80c4960 0x7fb2a8071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 -- 172.21.15.171:0/7880663 >> 172.21.15.171:0/7880663 conn(0x7fb2a80c7a80 msgr2=0x7fb2a80c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 -- 172.21.15.171:0/7880663 shutdown_connections 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.918+0000 7fb2adbce640 1 -- 172.21.15.171:0/7880663 wait complete. 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.919+0000 7fb2adbce640 1 Processor -- start 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.919+0000 7fb2adbce640 1 -- start start 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2adbce640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a8153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2adbce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2a8071cd0 0x7fb2a80c1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2adbce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2a80c4960 0x7fb2a8150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:09.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2adbce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80c4d40 con 0x7fb2a8071cd0 2024-04-25T01:01:09.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2adbce640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8152d50 con 0x7fb2a806fc20 2024-04-25T01:01:09.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2adbce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a80c6920 con 0x7fb2a80c4960 2024-04-25T01:01:09.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a77fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a8153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a77fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a8153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53584/0 (socket says 172.21.15.171:53584) 2024-04-25T01:01:09.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a77fe640 1 -- 172.21.15.171:0/225008144 learned_addr learned my addr 172.21.15.171:0/225008144 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:09.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a7fff640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2a80c4960 0x7fb2a8150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb29ffff640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2a8071cd0 0x7fb2a80c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:09.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a77fe640 1 -- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2a80c4960 msgr2=0x7fb2a8150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a77fe640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2a80c4960 0x7fb2a8150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.920+0000 7fb2a77fe640 1 -- 172.21.15.171:0/225008144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2a8071cd0 msgr2=0x7fb2a80c1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a77fe640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2a8071cd0 0x7fb2a80c1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a77fe640 1 -- 172.21.15.171:0/225008144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a81eb8e0 con 0x7fb2a806fc20 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a77fe640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a8153ba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e9d0 tx=0x7fb29800ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29800de60 con 0x7fb2a806fc20 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb298010440 con 0x7fb2a806fc20 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298010830 con 0x7fb2a806fc20 2024-04-25T01:01:09.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a8150000 con 0x7fb2a806fc20 2024-04-25T01:01:09.933 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a80c1660 con 0x7fb2a806fc20 2024-04-25T01:01:09.933 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2980109d0 con 0x7fb2a806fc20 2024-04-25T01:01:09.933 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.921+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fb298018db0 con 0x7fb2a806fc20 2024-04-25T01:01:09.933 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.922+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a8075550 con 0x7fb2a806fc20 2024-04-25T01:01:09.933 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.925+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb298037dd0 con 0x7fb2a806fc20 2024-04-25T01:01:09.968 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:09.968+0000 7fb9a7523040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:01:09.997 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:09.996+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5bfc9318-9d58-456c-81de-d3e7d2231fb1", "id": 4} v 0) -- 0x7fb2a80ad010 con 0x7fb2a806fc20 2024-04-25T01:01:10.003 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.002+0000 7fb2a4ff9640 1 -- 172.21.15.171:0/225008144 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5bfc9318-9d58-456c-81de-d3e7d2231fb1", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fb298026070 con 0x7fb2a806fc20 2024-04-25T01:01:10.003 INFO:teuthology.orchestra.run.smithi171.stdout:4 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 msgr2=0x7fb2a8153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a8153ba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e9d0 tx=0x7fb29800ee90 comp rx=0 tx=0).stop 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 shutdown_connections 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2a80c4960 0x7fb2a8150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2a8071cd0 0x7fb2a80c1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 --2- 172.21.15.171:0/225008144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a806fc20 0x7fb2a8153ba0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 >> 172.21.15.171:0/225008144 conn(0x7fb2a80c7a80 msgr2=0x7fb2a806da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 shutdown_connections 2024-04-25T01:01:10.004 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.003+0000 7fb2adbce640 1 -- 172.21.15.171:0/225008144 wait complete. 2024-04-25T01:01:10.013 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:10.013+0000 7fb9a7523040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:01:10.014 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 62b60dcc-6111-4df1-a716-f73967a074a2 5 2024-04-25T01:01:10.113 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.112+0000 7f97ae522640 1 Processor -- start 2024-04-25T01:01:10.113 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.112+0000 7f97ae522640 1 -- start start 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97ae522640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 0x7f97a8070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97ae522640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 0x7f97a80c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97ae522640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a8070cc0 con 0x7f97a80c4310 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97ae522640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f97a81541b0 con 0x7f97a80a82e0 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97ae522640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f97a80c8790 con 0x7f97a80703a0 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 0x7f97a8070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a77fe640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 0x7f97a80c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97acabf640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f97a80c4310 0x7f97a8156760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:60488/0 (socket says 172.21.15.171:60488) 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 0x7f97a8070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40314/0 (socket says 172.21.15.171:40314) 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a77fe640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 0x7f97a80c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53586/0 (socket says 172.21.15.171:53586) 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97acabf640 1 -- 172.21.15.171:0/3477939051 learned_addr learned my addr 172.21.15.171:0/3477939051 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-04-25T01:01:10.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a7fff640 1 -- 172.21.15.171:0/3477939051 >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 msgr2=0x7f97a80c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a7fff640 1 --2- 172.21.15.171:0/3477939051 >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 0x7f97a80c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a7fff640 1 -- 172.21.15.171:0/3477939051 >> v1:172.21.15.171:6789/0 conn(0x7f97a80c4310 legacy=0x7f97a8156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a7fff640 1 -- 172.21.15.171:0/3477939051 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80c3560 con 0x7f97a80703a0 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.113+0000 7f97a77fe640 1 --2- 172.21.15.171:0/3477939051 >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 0x7f97a80c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.114+0000 7f97a7fff640 1 --2- 172.21.15.171:0/3477939051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 0x7f97a8070780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097a0 tx=0x7f979c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f37b16ec5e6e1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.114+0000 7f97a6ffd640 1 -- 172.21.15.171:0/3477939051 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c037070 con 0x7f97a80703a0 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.114+0000 7f97a6ffd640 1 -- 172.21.15.171:0/3477939051 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c032440 con 0x7f97a80703a0 2024-04-25T01:01:10.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.114+0000 7f97a6ffd640 1 -- 172.21.15.171:0/3477939051 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c032810 con 0x7f97a80703a0 2024-04-25T01:01:10.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 -- 172.21.15.171:0/3477939051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 msgr2=0x7f97a8070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 --2- 172.21.15.171:0/3477939051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 0x7f97a8070780 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097a0 tx=0x7f979c02a180 comp rx=0 tx=0).stop 2024-04-25T01:01:10.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 -- 172.21.15.171:0/3477939051 shutdown_connections 2024-04-25T01:01:10.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 --2- 172.21.15.171:0/3477939051 >> v2:172.21.15.171:3300/0 conn(0x7f97a80a82e0 0x7f97a80c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 --2- 172.21.15.171:0/3477939051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80703a0 0x7f97a8070780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 -- 172.21.15.171:0/3477939051 >> 172.21.15.171:0/3477939051 conn(0x7f97a80aa7d0 msgr2=0x7f97a80aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 -- 172.21.15.171:0/3477939051 shutdown_connections 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.115+0000 7f97ae522640 1 -- 172.21.15.171:0/3477939051 wait complete. 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.116+0000 7f97ae522640 1 Processor -- start 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.116+0000 7f97ae522640 1 -- start start 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97ae522640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 0x7f97a8154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97ae522640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97a80a82e0 0x7f97a81eb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97ae522640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80c4310 0x7f97a80adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97ae522640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97a806ea20 con 0x7f97a80a82e0 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97ae522640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8153d20 con 0x7f97a80703a0 2024-04-25T01:01:10.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97ae522640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f97a80c4790 con 0x7f97a80c4310 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 0x7f97a8154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80c4310 0x7f97a80adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 0x7f97a8154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53602/0 (socket says 172.21.15.171:53602) 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97a7fff640 1 -- 172.21.15.171:0/491198032 learned_addr learned my addr 172.21.15.171:0/491198032 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97a77fe640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97a80a82e0 0x7f97a81eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 -- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 msgr2=0x7f97a8154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 0x7f97a8154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 -- 172.21.15.171:0/491198032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97a80a82e0 msgr2=0x7f97a81eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97a80a82e0 0x7f97a81eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 -- 172.21.15.171:0/491198032 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a81eadd0 con 0x7f97a80c4310 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97a77fe640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97a80a82e0 0x7f97a81eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97a7fff640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 0x7f97a8154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:10.121 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.117+0000 7f97acabf640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80c4310 0x7f97a80adfd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97a000caa0 tx=0x7f97a000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0017070 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a000ab40 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a806dd00 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0012400 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a81eb050 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97a001a070 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.118+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f97a001cc40 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.119+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a8075550 con 0x7f97a80c4310 2024-04-25T01:01:10.122 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.121+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f97a81eadd0 con 0x7f97a80c4310 2024-04-25T01:01:10.188 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:10.187+0000 7fb9a7523040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:01:10.191 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.191+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "62b60dcc-6111-4df1-a716-f73967a074a2", "id": 5} v 0) -- 0x7f97a80ac880 con 0x7f97a80c4310 2024-04-25T01:01:10.197 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.196+0000 7f97a4ff9640 1 -- 172.21.15.171:0/491198032 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62b60dcc-6111-4df1-a716-f73967a074a2", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f97a001c7f0 con 0x7f97a80c4310 2024-04-25T01:01:10.197 INFO:teuthology.orchestra.run.smithi171.stdout:5 2024-04-25T01:01:10.198 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80c4310 msgr2=0x7f97a80adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.198 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80c4310 0x7f97a80adfd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97a000caa0 tx=0x7f97a000cf60 comp rx=0 tx=0).stop 2024-04-25T01:01:10.198 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 shutdown_connections 2024-04-25T01:01:10.198 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97a80c4310 0x7f97a80adfd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.198 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97a80a82e0 0x7f97a81eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.199 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 --2- 172.21.15.171:0/491198032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97a80703a0 0x7f97a8154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.199 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 >> 172.21.15.171:0/491198032 conn(0x7f97a80aa7d0 msgr2=0x7f97a8150d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.199 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 shutdown_connections 2024-04-25T01:01:10.199 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.197+0000 7f97ae522640 1 -- 172.21.15.171:0/491198032 wait complete. 2024-04-25T01:01:10.209 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e 6 2024-04-25T01:01:10.244 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:10.243+0000 7fb9a7523040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:01:10.298 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:10.298+0000 7fb9a7523040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:01:10.306 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.305+0000 7f62af8f5640 1 Processor -- start 2024-04-25T01:01:10.306 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.305+0000 7f62af8f5640 1 -- start start 2024-04-25T01:01:10.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62af8f5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f62a8152a70 0x7f62a80c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62af8f5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f62a80c4a50 0x7f62a80a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62af8f5640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a8152e50 con 0x7f62a80a7230 2024-04-25T01:01:10.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62af8f5640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f62a81560e0 con 0x7f62a80c4a50 2024-04-25T01:01:10.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62af8f5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f62a81554d0 con 0x7f62a8152a70 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62ad691640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f62a8152a70 0x7f62a80c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62ad691640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f62a8152a70 0x7f62a80c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53604/0 (socket says 172.21.15.171:53604) 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.306+0000 7f62ad691640 1 -- 172.21.15.171:0/3205419726 learned_addr learned my addr 172.21.15.171:0/3205419726 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.307+0000 7f62ace90640 1 --2- 172.21.15.171:0/3205419726 >> v2:172.21.15.129:3300/0 conn(0x7f62a80c4a50 0x7f62a80a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.307+0000 7f62ad691640 1 -- 172.21.15.171:0/3205419726 >> v2:172.21.15.129:3300/0 conn(0x7f62a80c4a50 msgr2=0x7f62a80a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.307+0000 7f62ad691640 1 --2- 172.21.15.171:0/3205419726 >> v2:172.21.15.129:3300/0 conn(0x7f62a80c4a50 0x7f62a80a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.307+0000 7f62ad691640 1 -- 172.21.15.171:0/3205419726 >> v1:172.21.15.129:6789/0 conn(0x7f62a80a7230 legacy=0x7f62a8158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.307+0000 7f62ad691640 1 -- 172.21.15.171:0/3205419726 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a81557b0 con 0x7f62a8152a70 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.307+0000 7f62ad691640 1 --2- 172.21.15.171:0/3205419726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a80c4510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f62a80aa8b0 tx=0x7f629802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc01548a46b44be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f629ffff640 1 -- 172.21.15.171:0/3205419726 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298037070 con 0x7f62a8152a70 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f629ffff640 1 -- 172.21.15.171:0/3205419726 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629802ae20 con 0x7f62a8152a70 2024-04-25T01:01:10.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f629ffff640 1 -- 172.21.15.171:0/3205419726 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298032660 con 0x7f62a8152a70 2024-04-25T01:01:10.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 -- 172.21.15.171:0/3205419726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 msgr2=0x7f62a80c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 --2- 172.21.15.171:0/3205419726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a80c4510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f62a80aa8b0 tx=0x7f629802a110 comp rx=0 tx=0).stop 2024-04-25T01:01:10.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 -- 172.21.15.171:0/3205419726 shutdown_connections 2024-04-25T01:01:10.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 --2- 172.21.15.171:0/3205419726 >> v2:172.21.15.129:3300/0 conn(0x7f62a80c4a50 0x7f62a80a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 --2- 172.21.15.171:0/3205419726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a80c4510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 -- 172.21.15.171:0/3205419726 >> 172.21.15.171:0/3205419726 conn(0x7f62a806ca60 msgr2=0x7f62a80c7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 -- 172.21.15.171:0/3205419726 shutdown_connections 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.308+0000 7f62af8f5640 1 -- 172.21.15.171:0/3205419726 wait complete. 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.309+0000 7f62af8f5640 1 Processor -- start 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.309+0000 7f62af8f5640 1 -- start start 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62af8f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62a80a7230 0x7f62a8072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62af8f5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 0x7f62a81eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62af8f5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a81eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62af8f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80c1850 con 0x7f62a80a7230 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62af8f5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8155d70 con 0x7f62a8152a70 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62af8f5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f62a8159130 con 0x7f62a80c4a50 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ace90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 0x7f62a81eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ade92640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a81eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ace90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 0x7f62a81eae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40328/0 (socket says 172.21.15.171:40328) 2024-04-25T01:01:10.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ace90640 1 -- 172.21.15.171:0/1695873314 learned_addr learned my addr 172.21.15.171:0/1695873314 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ad691640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62a80a7230 0x7f62a8072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ade92640 1 -- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 msgr2=0x7f62a81eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ade92640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 0x7f62a81eae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ade92640 1 -- 172.21.15.171:0/1695873314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62a80a7230 msgr2=0x7f62a8072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.310+0000 7f62ade92640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62a80a7230 0x7f62a8072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f62ade92640 1 -- 172.21.15.171:0/1695873314 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a81ee370 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f62ace90640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 0x7f62a81eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f62ade92640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a81eb380 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c7b0 tx=0x7f62a000cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0016020 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a000aa50 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a81ee650 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a000ce60 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f62a0019020 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81eec10 con 0x7f62a8152a70 2024-04-25T01:01:10.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.311+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f62a001b400 con 0x7f62a8152a70 2024-04-25T01:01:10.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.312+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a8075550 con 0x7f62a8152a70 2024-04-25T01:01:10.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.314+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f62a0020020 con 0x7f62a8152a70 2024-04-25T01:01:10.385 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.384+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e", "id": 6} v 0) -- 0x7f62a80c7cb0 con 0x7f62a8152a70 2024-04-25T01:01:10.392 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.391+0000 7f629dffb640 1 -- 172.21.15.171:0/1695873314 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f62a0004950 con 0x7f62a8152a70 2024-04-25T01:01:10.392 INFO:teuthology.orchestra.run.smithi171.stdout:6 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 msgr2=0x7f62a81eb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a81eb380 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c7b0 tx=0x7f62a000cc70 comp rx=0 tx=0).stop 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 shutdown_connections 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62a80c4a50 0x7f62a81eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62a8152a70 0x7f62a81eb380 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 --2- 172.21.15.171:0/1695873314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62a80a7230 0x7f62a8072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 >> 172.21.15.171:0/1695873314 conn(0x7f62a806ca60 msgr2=0x7f62a814e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 shutdown_connections 2024-04-25T01:01:10.393 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.392+0000 7f62af8f5640 1 -- 172.21.15.171:0/1695873314 wait complete. 2024-04-25T01:01:10.403 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 93678b87-0929-4133-ad43-1f15219e89e9 7 2024-04-25T01:01:10.414 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:10.414+0000 7fb9a7523040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:01:10.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.513+0000 7ff63aa76640 1 Processor -- start 2024-04-25T01:01:10.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.513+0000 7ff63aa76640 1 -- start start 2024-04-25T01:01:10.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff63aa76640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff634070690 0x7ff6340c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff63aa76640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 0x7ff6340a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff63aa76640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63406e940 con 0x7ff634070a70 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff63aa76640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff63406ef00 con 0x7ff634070690 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff63aa76640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff63406ec20 con 0x7ff634153c60 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 0x7ff6340a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 0x7ff6340a7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40340/0 (socket says 172.21.15.171:40340) 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 -- 172.21.15.171:0/2685572727 learned_addr learned my addr 172.21.15.171:0/2685572727 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff638812640 1 --2- 172.21.15.171:0/2685572727 >> v2:172.21.15.129:3300/0 conn(0x7ff634070690 0x7ff6340c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 -- 172.21.15.171:0/2685572727 >> v2:172.21.15.129:3300/0 conn(0x7ff634070690 msgr2=0x7ff6340c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 --2- 172.21.15.171:0/2685572727 >> v2:172.21.15.129:3300/0 conn(0x7ff634070690 0x7ff6340c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 -- 172.21.15.171:0/2685572727 >> v1:172.21.15.129:6789/0 conn(0x7ff634070a70 legacy=0x7ff634150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.514+0000 7ff633fff640 1 -- 172.21.15.171:0/2685572727 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6340c35b0 con 0x7ff634153c60 2024-04-25T01:01:10.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.515+0000 7ff633fff640 1 --2- 172.21.15.171:0/2685572727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 0x7ff6340a7e70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff620009870 tx=0x7ff62002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732f926da3b01a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.515+0000 7ff6337fe640 1 -- 172.21.15.171:0/2685572727 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620037070 con 0x7ff634153c60 2024-04-25T01:01:10.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.515+0000 7ff6337fe640 1 -- 172.21.15.171:0/2685572727 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff620032440 con 0x7ff634153c60 2024-04-25T01:01:10.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.515+0000 7ff6337fe640 1 -- 172.21.15.171:0/2685572727 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620032810 con 0x7ff634153c60 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.515+0000 7ff63aa76640 1 -- 172.21.15.171:0/2685572727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 msgr2=0x7ff6340a7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.515+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2685572727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 0x7ff6340a7e70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff620009870 tx=0x7ff62002a1c0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.516+0000 7ff63aa76640 1 -- 172.21.15.171:0/2685572727 shutdown_connections 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.516+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2685572727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634153c60 0x7ff6340a7e70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.516+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2685572727 >> v2:172.21.15.129:3300/0 conn(0x7ff634070690 0x7ff6340c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.516+0000 7ff63aa76640 1 -- 172.21.15.171:0/2685572727 >> 172.21.15.171:0/2685572727 conn(0x7ff6340c8a90 msgr2=0x7ff6340c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.516+0000 7ff63aa76640 1 -- 172.21.15.171:0/2685572727 shutdown_connections 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.516+0000 7ff63aa76640 1 -- 172.21.15.171:0/2685572727 wait complete. 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.517+0000 7ff63aa76640 1 Processor -- start 2024-04-25T01:01:10.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.517+0000 7ff63aa76640 1 -- start start 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.517+0000 7ff63aa76640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 0x7ff634147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.517+0000 7ff63aa76640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff634070a70 0x7ff6341ea870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff63aa76640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff634153c60 0x7ff634145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff63aa76640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6340a83b0 con 0x7ff634153c60 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff63aa76640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341540c0 con 0x7ff634070a70 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff63aa76640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff634150c30 con 0x7ff634070690 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 0x7ff634147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 0x7ff634147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:40354/0 (socket says 172.21.15.171:40354) 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 -- 172.21.15.171:0/2853896633 learned_addr learned my addr 172.21.15.171:0/2853896633 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff633fff640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff634070a70 0x7ff6341ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff639013640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff634153c60 0x7ff634145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 -- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff634070a70 msgr2=0x7ff6341ea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff634070a70 0x7ff6341ea870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 -- 172.21.15.171:0/2853896633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff634153c60 msgr2=0x7ff634145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff634153c60 0x7ff634145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 -- 172.21.15.171:0/2853896633 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6340c7170 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff638812640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 0x7ff634147bf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff62800e9e0 tx=0x7ff62800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628017070 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62800c9d0 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62800cdc0 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6341edf10 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6341ee490 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff62801a070 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.518+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff628006660 con 0x7ff634070690 2024-04-25T01:01:10.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.519+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63406bb20 con 0x7ff634070690 2024-04-25T01:01:10.523 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.521+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff628005990 con 0x7ff634070690 2024-04-25T01:01:10.594 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.593+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "93678b87-0929-4133-ad43-1f15219e89e9", "id": 7} v 0) -- 0x7ff634075550 con 0x7ff634070690 2024-04-25T01:01:10.599 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.599+0000 7ff6317fa640 1 -- 172.21.15.171:0/2853896633 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93678b87-0929-4133-ad43-1f15219e89e9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff628005c40 con 0x7ff634070690 2024-04-25T01:01:10.600 INFO:teuthology.orchestra.run.smithi171.stdout:7 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 msgr2=0x7ff634147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 0x7ff634147bf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff62800e9e0 tx=0x7ff62800eea0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 shutdown_connections 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff634153c60 0x7ff634145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff634070a70 0x7ff6341ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 --2- 172.21.15.171:0/2853896633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff634070690 0x7ff634147bf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 >> 172.21.15.171:0/2853896633 conn(0x7ff6340c8a90 msgr2=0x7ff63414ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 shutdown_connections 2024-04-25T01:01:10.601 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:01:10.600+0000 7ff63aa76640 1 -- 172.21.15.171:0/2853896633 wait complete. 2024-04-25T01:01:10.602 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:10.602+0000 7f4f20e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:01:10.611 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T01:01:10.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 0 2024-04-25T01:01:10.613 INFO:tasks.ceph.osd.0:Started 2024-04-25T01:01:10.613 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T01:01:10.614 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 1 2024-04-25T01:01:10.618 INFO:tasks.ceph.osd.1:Started 2024-04-25T01:01:10.618 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T01:01:10.618 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 2 2024-04-25T01:01:10.620 INFO:tasks.ceph.osd.2:Started 2024-04-25T01:01:10.620 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T01:01:10.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 3 2024-04-25T01:01:10.623 INFO:tasks.ceph.osd.3:Started 2024-04-25T01:01:10.623 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T01:01:10.623 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 4 2024-04-25T01:01:10.625 INFO:tasks.ceph.osd.4:Started 2024-04-25T01:01:10.625 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T01:01:10.625 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 5 2024-04-25T01:01:10.627 INFO:tasks.ceph.osd.5:Started 2024-04-25T01:01:10.627 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T01:01:10.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 6 2024-04-25T01:01:10.629 INFO:tasks.ceph.osd.6:Started 2024-04-25T01:01:10.629 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T01:01:10.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 7 2024-04-25T01:01:10.631 INFO:tasks.ceph.osd.7:Started 2024-04-25T01:01:10.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:10.726 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:10.726+0000 7f4f20e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:01:10.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.748+0000 7ff4fe71d640 1 Processor -- start 2024-04-25T01:01:10.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.748+0000 7ff4fe71d640 1 -- start start 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4fe71d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4fe71d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f80c9860 con 0x7ff4f80c5520 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4fe71d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f8156690 con 0x7ff4f81561e0 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4fe71d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f80c5a70 con 0x7ff4f8152fa0 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f77fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff4f80c5520 0x7ff4f80c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34404/0 (socket says 172.21.15.129:34404) 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f77fe640 1 -- 172.21.15.129:0/823867597 learned_addr learned my addr 172.21.15.129:0/823867597 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:10.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f7fff640 1 --2- 172.21.15.129:0/823867597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720140517 0 0) 0x7ff4e8001110 con 0x7ff4f80c5520 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4e0003520 con 0x7ff4f80c5520 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3615990454 0 0) 0x7ff4f8156690 con 0x7ff4f81561e0 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.749+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4e8001110 con 0x7ff4f81561e0 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.750+0000 7ff4f7fff640 1 -- 172.21.15.129:0/823867597 >> v1:172.21.15.171:6789/0 conn(0x7ff4f81561e0 legacy=0x7ff4f80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.750+0000 7ff4f7fff640 1 -- 172.21.15.129:0/823867597 >> v1:172.21.15.129:6789/0 conn(0x7ff4f80c5520 legacy=0x7ff4f80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.750+0000 7ff4f7fff640 1 -- 172.21.15.129:0/823867597 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f80c5d50 con 0x7ff4f8152fa0 2024-04-25T01:01:10.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.750+0000 7ff4f7fff640 1 --2- 172.21.15.129:0/823867597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80ca2e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0097d0 tx=0x7ff4ec02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5eea89bb773747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.750+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec037070 con 0x7ff4f8152fa0 2024-04-25T01:01:10.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.750+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ec02ae50 con 0x7ff4f8152fa0 2024-04-25T01:01:10.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 -- 172.21.15.129:0/823867597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 msgr2=0x7ff4f80ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4f6ffd640 1 -- 172.21.15.129:0/823867597 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec037070 con 0x7ff4f8152fa0 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 --2- 172.21.15.129:0/823867597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80ca2e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0097d0 tx=0x7ff4ec02a100 comp rx=0 tx=0).stop 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 -- 172.21.15.129:0/823867597 shutdown_connections 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 --2- 172.21.15.129:0/823867597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80ca2e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 -- 172.21.15.129:0/823867597 >> 172.21.15.129:0/823867597 conn(0x7ff4f806c400 msgr2=0x7ff4f806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 -- 172.21.15.129:0/823867597 shutdown_connections 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.751+0000 7ff4fe71d640 1 -- 172.21.15.129:0/823867597 wait complete. 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.752+0000 7ff4fe71d640 1 Processor -- start 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.752+0000 7ff4fe71d640 1 -- start start 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fe71d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 0x7ff4f80c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fe71d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80c0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fe71d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4f81561e0 0x7ff4f80c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fe71d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f80c93d0 con 0x7ff4f80c5520 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fe71d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f80c6030 con 0x7ff4f81561e0 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fe71d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f80c9550 con 0x7ff4f8152fa0 2024-04-25T01:01:10.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 0x7ff4f80c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 0x7ff4f80c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57708/0 (socket says 172.21.15.129:57708) 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 -- 172.21.15.129:0/3314857601 learned_addr learned my addr 172.21.15.129:0/3314857601 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f77fe640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4fccba640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4f81561e0 0x7ff4f80c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 -- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 msgr2=0x7ff4f80c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80c0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 -- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4f81561e0 msgr2=0x7ff4f80c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4f81561e0 0x7ff4f80c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.753+0000 7ff4f7fff640 1 -- 172.21.15.129:0/3314857601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f80c1da0 con 0x7ff4f80c5520 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4f7fff640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 0x7ff4f80c1370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec035040 tx=0x7ff4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec037050 con 0x7ff4f80c5520 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ec043070 con 0x7ff4f80c5520 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec03e690 con 0x7ff4f80c5520 2024-04-25T01:01:10.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f80c2080 con 0x7ff4f80c5520 2024-04-25T01:01:10.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f80c18b0 con 0x7ff4f80c5520 2024-04-25T01:01:10.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4ec03e830 con 0x7ff4f80c5520 2024-04-25T01:01:10.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.754+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff4ec03eb20 con 0x7ff4f80c5520 2024-04-25T01:01:10.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.755+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f8075550 con 0x7ff4f80c5520 2024-04-25T01:01:10.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.757+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff4ec04d020 con 0x7ff4f80c5520 2024-04-25T01:01:10.771 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:10.770+0000 7f4f20e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:01:10.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.817+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4f80ca0c0 con 0x7ff4f80c5520 2024-04-25T01:01:10.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.818+0000 7ff4f4ff9640 1 -- 172.21.15.129:0/3314857601 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff4ec003e20 con 0x7ff4f80c5520 2024-04-25T01:01:10.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:10.818 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":10,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:10.595760+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:10.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 msgr2=0x7ff4f80c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 0x7ff4f80c1370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec035040 tx=0x7ff4ec0046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 shutdown_connections 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff4f8152fa0 0x7ff4f80c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4f81561e0 0x7ff4f80c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 --2- 172.21.15.129:0/3314857601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4f80c5520 0x7ff4f80c1370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 >> 172.21.15.129:0/3314857601 conn(0x7ff4f806c400 msgr2=0x7ff4f806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 shutdown_connections 2024-04-25T01:01:10.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:10.819+0000 7ff4fe71d640 1 -- 172.21.15.129:0/3314857601 wait complete. 2024-04-25T01:01:10.830 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T01:01:10.830 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T01:01:10.843 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:10.843+0000 7fb9a7523040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:01:10.938 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:10.938+0000 7f4f20e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:01:10.994 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:10.993+0000 7f4f20e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:01:11.047 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:11.046+0000 7f4f20e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:01:11.100 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.100+0000 7fb9a7523040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:01:11.151 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.151+0000 7fb9a7523040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:01:11.157 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:11.157+0000 7f4f20e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:01:11.210 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:01:11.209+0000 7fce9541d640 -1 Falling back to public interface 2024-04-25T01:01:11.223 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:01:11.222+0000 7f500f721640 -1 Falling back to public interface 2024-04-25T01:01:11.231 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:01:11.230+0000 7f7d4dedd640 -1 Falling back to public interface 2024-04-25T01:01:11.231 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:01:11.230+0000 7fd1ab6df640 -1 Falling back to public interface 2024-04-25T01:01:11.232 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:01:11.231+0000 7f3093484640 -1 Falling back to public interface 2024-04-25T01:01:11.234 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:01:11.234+0000 7f1f4a420640 -1 Falling back to public interface 2024-04-25T01:01:11.241 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:01:11.241+0000 7fd7e680d640 -1 Falling back to public interface 2024-04-25T01:01:11.261 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:01:11.261+0000 7f7f01023640 -1 Falling back to public interface 2024-04-25T01:01:11.273 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.272+0000 7fb9a7523040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:01:11.541 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:11.540+0000 7f4f20e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:01:11.552 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.552+0000 7fb9a7523040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:01:11.600 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.599+0000 7fb9a7523040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:01:11.634 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:11.726 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.725+0000 7f2a305c4640 1 Processor -- start 2024-04-25T01:01:11.726 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.725+0000 7f2a305c4640 1 -- start start 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a305c4640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2a280c2010 0x7f2a280c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a305c4640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a280c2930 con 0x7f2a2814f420 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a305c4640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2814f800 con 0x7f2a280c6ca0 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a305c4640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f2a28155c40 con 0x7f2a280c2010 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a2eb61640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2a280c2010 0x7f2a280c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a2e360640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f2a2814f420 0x7f2a280c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:34408/0 (socket says 172.21.15.129:34408) 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a2e360640 1 -- 172.21.15.129:0/3734297648 learned_addr learned my addr 172.21.15.129:0/3734297648 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:11.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.726+0000 7f2a2eb61640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2a280c2010 0x7f2a280c23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57720/0 (socket says 172.21.15.129:57720) 2024-04-25T01:01:11.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2eb61640 1 -- 172.21.15.129:0/3734297648 >> v1:172.21.15.171:6789/0 conn(0x7f2a280c6ca0 legacy=0x7f2a280a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:11.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2eb61640 1 -- 172.21.15.129:0/3734297648 >> v1:172.21.15.129:6789/0 conn(0x7f2a2814f420 legacy=0x7f2a280c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:11.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2eb61640 1 -- 172.21.15.129:0/3734297648 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a28155330 con 0x7f2a280c2010 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2eb61640 1 --2- 172.21.15.129:0/3734297648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a280c23f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009d90 tx=0x7f2a2002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5830087c923b8a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2d35e640 1 -- 172.21.15.129:0/3734297648 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20031600 con 0x7f2a280c2010 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2d35e640 1 -- 172.21.15.129:0/3734297648 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a200317a0 con 0x7f2a280c2010 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.727+0000 7f2a2d35e640 1 -- 172.21.15.129:0/3734297648 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20031b70 con 0x7f2a280c2010 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 -- 172.21.15.129:0/3734297648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 msgr2=0x7f2a280c23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 --2- 172.21.15.129:0/3734297648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a280c23f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009d90 tx=0x7f2a2002a5d0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 -- 172.21.15.129:0/3734297648 shutdown_connections 2024-04-25T01:01:11.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 --2- 172.21.15.129:0/3734297648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a280c23f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.730 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 -- 172.21.15.129:0/3734297648 >> 172.21.15.129:0/3734297648 conn(0x7f2a280c8be0 msgr2=0x7f2a280c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:11.730 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 -- 172.21.15.129:0/3734297648 shutdown_connections 2024-04-25T01:01:11.730 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.728+0000 7f2a305c4640 1 -- 172.21.15.129:0/3734297648 wait complete. 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.729+0000 7f2a305c4640 1 Processor -- start 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.729+0000 7f2a305c4640 1 -- start start 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a305c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a281ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a305c4640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2a280c6ca0 0x7f2a28147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a305c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a2814f420 0x7f2a281eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a305c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a281558d0 con 0x7f2a280c2010 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a305c4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2a281555f0 con 0x7f2a2814f420 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a305c4640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f2a280a84c0 con 0x7f2a280c6ca0 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a281ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a281ee870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57724/0 (socket says 172.21.15.129:57724) 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 -- 172.21.15.129:0/2872438111 learned_addr learned my addr 172.21.15.129:0/2872438111 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:11.732 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2eb61640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a2814f420 0x7f2a281eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2db5f640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2a280c6ca0 0x7f2a28147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 -- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2a280c6ca0 msgr2=0x7f2a28147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2a280c6ca0 0x7f2a28147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 -- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a2814f420 msgr2=0x7f2a281eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a2814f420 0x7f2a281eb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 -- 172.21.15.129:0/2872438111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a281e0320 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a2e360640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a281ee870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400c4d0 tx=0x7f2a2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.730+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24017070 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.731+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24005ce0 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.731+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24007270 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.731+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a281ef200 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.731+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2806db60 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.731+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a2401a070 con 0x7f2a280c2010 2024-04-25T01:01:11.733 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.731+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2a2401ccd0 con 0x7f2a280c2010 2024-04-25T01:01:11.734 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.732+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a28075570 con 0x7f2a280c2010 2024-04-25T01:01:11.734 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.734+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a280c2930 con 0x7f2a280c2010 2024-04-25T01:01:11.756 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.755+0000 7fb9a7523040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:01:11.796 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.796+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a28153650 con 0x7f2a280c2010 2024-04-25T01:01:11.797 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.796+0000 7f2a1effd640 1 -- 172.21.15.129:0/2872438111 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2a24007410 con 0x7f2a280c2010 2024-04-25T01:01:11.797 INFO:teuthology.misc.health.smithi129.stdout: 2024-04-25T01:01:11.797 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":10,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:10.595760+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 msgr2=0x7f2a281ee870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a281ee870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400c4d0 tx=0x7f2a2400c990 comp rx=0 tx=0).stop 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 shutdown_connections 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2a280c6ca0 0x7f2a28147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a2814f420 0x7f2a281eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 --2- 172.21.15.129:0/2872438111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a280c2010 0x7f2a281ee870 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:11.799 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 >> 172.21.15.129:0/2872438111 conn(0x7f2a280c8be0 msgr2=0x7f2a281546f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:11.800 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 shutdown_connections 2024-04-25T01:01:11.800 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:11.797+0000 7f2a305c4640 1 -- 172.21.15.129:0/2872438111 wait complete. 2024-04-25T01:01:11.806 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:11.806+0000 7f4f20e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:01:11.807 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T01:01:11.859 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:11.858+0000 7f4f20e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:01:11.892 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.891+0000 7fb9a7523040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:01:11.941 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:11.940+0000 7fb9a7523040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:01:11.973 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:11.973+0000 7f4f20e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:01:12.010 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:12.010+0000 7fb9a7523040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:01:12.056 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:12.056+0000 7fb9a7523040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:01:12.251 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.251+0000 7f4f20e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:01:12.299 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.299+0000 7f4f20e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:01:12.455 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.455+0000 7f4f20e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:01:12.530 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:12.529+0000 7fb9a7523040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:01:12.593 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.592+0000 7f4f20e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:01:12.643 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.642+0000 7f4f20e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:01:12.647 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:12.647+0000 7fb9a7523040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:01:12.718 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.718+0000 7f4f20e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:01:12.762 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:12.762+0000 7f4f20e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:01:13.240 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:13.240+0000 7f4f20e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:01:13.355 INFO:tasks.ceph.mgr.y.smithi129.stderr:2024-04-25T01:01:13.355+0000 7f4f20e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:01:13.666 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:13.665+0000 7fb9480aa640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T01:01:13.666 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:01:13.665+0000 7fb9480aa640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T01:01:14.714 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:01:14.713+0000 7f500f721640 -1 osd.7 0 log_to_monitors true 2024-04-25T01:01:14.732 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:01:14.732+0000 7fce9541d640 -1 osd.0 0 log_to_monitors true 2024-04-25T01:01:14.745 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:01:14.744+0000 7f1f4a420640 -1 osd.1 0 log_to_monitors true 2024-04-25T01:01:14.750 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:01:14.750+0000 7f3093484640 -1 osd.3 0 log_to_monitors true 2024-04-25T01:01:14.751 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:01:14.751+0000 7fd1ab6df640 -1 osd.5 0 log_to_monitors true 2024-04-25T01:01:14.757 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:01:14.757+0000 7fd7e680d640 -1 osd.2 0 log_to_monitors true 2024-04-25T01:01:14.767 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:01:14.767+0000 7f7d4dedd640 -1 osd.4 0 log_to_monitors true 2024-04-25T01:01:14.790 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:01:14.789+0000 7f7f01023640 -1 osd.6 0 log_to_monitors true 2024-04-25T01:01:18.314 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:01:18.313+0000 7fce91bc7640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T01:01:18.330 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:01:18.330+0000 7fce891d3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:18.371 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:01:18.371+0000 7f308fc2e640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T01:01:18.393 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:01:18.393+0000 7f308723a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:18.614 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:18.719 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.719+0000 7f275313b640 1 Processor -- start 2024-04-25T01:01:18.719 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.719+0000 7f275313b640 1 -- start start 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f275313b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f274c152a10 0x7f274c0c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f275313b640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f274c0c8b10 0x7f274c0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f275313b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c0caa40 con 0x7f274c1588b0 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f275313b640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f274c0c5c60 con 0x7f274c152a10 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f275313b640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f274c0ca580 con 0x7f274c0c8b10 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f2750ed7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f274c152a10 0x7f274c0c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f27516d8640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f274c1588b0 0x7f274c06b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55492/0 (socket says 172.21.15.129:55492) 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f274bfff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f274c0c8b10 0x7f274c0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:18.721 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f27516d8640 1 -- 172.21.15.129:0/1172363359 learned_addr learned my addr 172.21.15.129:0/1172363359 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:18.722 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f274b7fe640 1 -- 172.21.15.129:0/1172363359 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477402337 0 0) 0x7f274c0caa40 con 0x7f274c1588b0 2024-04-25T01:01:18.722 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f274bfff640 1 -- 172.21.15.129:0/1172363359 >> v2:172.21.15.129:3300/0 conn(0x7f274c152a10 msgr2=0x7f274c0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:18.722 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f274bfff640 1 --2- 172.21.15.129:0/1172363359 >> v2:172.21.15.129:3300/0 conn(0x7f274c152a10 0x7f274c0c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:18.722 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f274bfff640 1 -- 172.21.15.129:0/1172363359 >> v1:172.21.15.129:6789/0 conn(0x7f274c1588b0 legacy=0x7f274c06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:18.722 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.720+0000 7f274bfff640 1 -- 172.21.15.129:0/1172363359 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0c8fa0 con 0x7f274c0c8b10 2024-04-25T01:01:18.723 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.721+0000 7f274bfff640 1 --2- 172.21.15.129:0/1172363359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c0c8b10 0x7f274c0ca040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f274c08c730 tx=0x7f273802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975eaf1a34fb93f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:18.723 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.722+0000 7f274b7fe640 1 -- 172.21.15.129:0/1172363359 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738037070 con 0x7f274c0c8b10 2024-04-25T01:01:18.723 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.722+0000 7f274b7fe640 1 -- 172.21.15.129:0/1172363359 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2738032440 con 0x7f274c0c8b10 2024-04-25T01:01:18.723 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.722+0000 7f274b7fe640 1 -- 172.21.15.129:0/1172363359 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273802ae60 con 0x7f274c0c8b10 2024-04-25T01:01:18.723 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 -- 172.21.15.129:0/1172363359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c0c8b10 msgr2=0x7f274c0ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 --2- 172.21.15.129:0/1172363359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c0c8b10 0x7f274c0ca040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f274c08c730 tx=0x7f273802a110 comp rx=0 tx=0).stop 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 -- 172.21.15.129:0/1172363359 shutdown_connections 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 --2- 172.21.15.129:0/1172363359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c0c8b10 0x7f274c0ca040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 --2- 172.21.15.129:0/1172363359 >> v2:172.21.15.129:3300/0 conn(0x7f274c152a10 0x7f274c0c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 -- 172.21.15.129:0/1172363359 >> 172.21.15.129:0/1172363359 conn(0x7f274c0ac700 msgr2=0x7f274c0acb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 -- 172.21.15.129:0/1172363359 shutdown_connections 2024-04-25T01:01:18.724 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.723+0000 7f275313b640 1 -- 172.21.15.129:0/1172363359 wait complete. 2024-04-25T01:01:18.725 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.724+0000 7f275313b640 1 Processor -- start 2024-04-25T01:01:18.725 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.724+0000 7f275313b640 1 -- start start 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f275313b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c152a10 0x7f274c1ee840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f275313b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 0x7f274c0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f275313b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f274c153a00 0x7f274c147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f275313b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0c9840 con 0x7f274c1588b0 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f275313b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0c9560 con 0x7f274c152a10 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f275313b640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0c9b20 con 0x7f274c153a00 2024-04-25T01:01:18.727 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f274bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 0x7f274c0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f274bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 0x7f274c0aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41004/0 (socket says 172.21.15.129:41004) 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f274bfff640 1 -- 172.21.15.129:0/633911119 learned_addr learned my addr 172.21.15.129:0/633911119 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.725+0000 7f2750ed7640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c152a10 0x7f274c1ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f27516d8640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f274c153a00 0x7f274c147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f274bfff640 1 -- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f274c153a00 msgr2=0x7f274c147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f274bfff640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f274c153a00 0x7f274c147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f274bfff640 1 -- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c152a10 msgr2=0x7f274c1ee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f274bfff640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c152a10 0x7f274c1ee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f274bfff640 1 -- 172.21.15.129:0/633911119 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c1eed80 con 0x7f274c1588b0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f274bfff640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 0x7f274c0aa9f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2738000c00 tx=0x7f27380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738043070 con 0x7f274c1588b0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2738038040 con 0x7f274c1588b0 2024-04-25T01:01:18.728 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738037050 con 0x7f274c1588b0 2024-04-25T01:01:18.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c0aaf30 con 0x7f274c1588b0 2024-04-25T01:01:18.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.726+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c06bc70 con 0x7f274c1588b0 2024-04-25T01:01:18.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.728+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f2738046070 con 0x7f274c1588b0 2024-04-25T01:01:18.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.728+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c06beb0 con 0x7f274c1588b0 2024-04-25T01:01:18.729 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.728+0000 7f27497fa640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f273006bea0 0x7f273006e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:18.730 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.729+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f27380aa090 con 0x7f274c1588b0 2024-04-25T01:01:18.730 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.729+0000 7f2750ed7640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f273006bea0 0x7f273006e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:18.730 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.729+0000 7f2750ed7640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f273006bea0 0x7f273006e360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2740009b40 tx=0x7f2740009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:18.735 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:18.734+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f274c06beb0 con 0x7f274c1588b0 2024-04-25T01:01:18.760 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:01:18.760+0000 7f7efca67640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T01:01:18.778 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:01:18.778+0000 7f7ef4887640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:18.819 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:01:18.819+0000 7f500b6b7640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T01:01:18.834 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:01:18.834+0000 7f50034d7640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:18.835 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:01:18.834+0000 7fd1a7675640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T01:01:18.837 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:01:18.837+0000 7f7d49e73640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T01:01:18.848 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:01:18.848+0000 7f7d41c93640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:18.848 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:01:18.848+0000 7fd19f495640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:19.030 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.030+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f274c075550 con 0x7f274c1588b0 2024-04-25T01:01:19.031 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.030+0000 7f27497fa640 1 -- 172.21.15.129:0/633911119 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f2738079240 con 0x7f274c1588b0 2024-04-25T01:01:19.031 INFO:teuthology.misc.health.smithi129.stdout: 2024-04-25T01:01:19.031 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":12,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:16.672076+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:19.033 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.032+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f273006bea0 msgr2=0x7f273006e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:19.033 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.032+0000 7f275313b640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f273006bea0 0x7f273006e360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2740009b40 tx=0x7f2740009190 comp rx=0 tx=0).stop 2024-04-25T01:01:19.033 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.032+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 msgr2=0x7f274c0aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.032+0000 7f275313b640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 0x7f274c0aa9f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2738000c00 tx=0x7f27380046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 shutdown_connections 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f273006bea0 0x7f273006e360 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f274c153a00 0x7f274c147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f274c1588b0 0x7f274c0aa9f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 --2- 172.21.15.129:0/633911119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f274c152a10 0x7f274c1ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 >> 172.21.15.129:0/633911119 conn(0x7f274c0ac700 msgr2=0x7f274c150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 shutdown_connections 2024-04-25T01:01:19.034 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:19.033+0000 7f275313b640 1 -- 172.21.15.129:0/633911119 wait complete. 2024-04-25T01:01:19.044 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T01:01:19.214 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:01:19.213+0000 7fd7e2198640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T01:01:19.228 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:01:19.228+0000 7fd7d9fb8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:19.234 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:01:19.234+0000 7f1f45d17640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T01:01:19.248 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:01:19.248+0000 7f1f3db37640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:01:25.851 DEBUG:teuthology.orchestra.run.smithi129:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:25.950 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.949+0000 7fc2884f5640 1 Processor -- start 2024-04-25T01:01:25.950 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.949+0000 7fc2884f5640 1 -- start start 2024-04-25T01:01:25.953 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.950+0000 7fc2884f5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc280152b50 0x7fc2800c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.953 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.950+0000 7fc2884f5640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc280156330 con 0x7fc2800be2c0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.950+0000 7fc2884f5640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2800c56c0 con 0x7fc2800c8bc0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.950+0000 7fc2884f5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fc2800c59a0 con 0x7fc280152b50 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.950+0000 7fc286a92640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc2800be2c0 0x7fc28015fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55500/0 (socket says 172.21.15.129:55500) 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.950+0000 7fc286a92640 1 -- 172.21.15.129:0/3208396263 learned_addr learned my addr 172.21.15.129:0/3208396263 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 --2- 172.21.15.129:0/3208396263 >> v2:172.21.15.171:3300/0 conn(0x7fc280152b50 0x7fc2800c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430761964 0 0) 0x7fc280156330 con 0x7fc2800be2c0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc26c003520 con 0x7fc2800be2c0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1647336677 0 0) 0x7fc26c003520 con 0x7fc2800be2c0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 >> v2:172.21.15.171:3300/0 conn(0x7fc280152b50 msgr2=0x7fc2800c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 --2- 172.21.15.129:0/3208396263 >> v2:172.21.15.171:3300/0 conn(0x7fc280152b50 0x7fc2800c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 >> v1:172.21.15.171:6789/0 conn(0x7fc2800c8bc0 legacy=0x7fc28006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800c5c80 con 0x7fc2800be2c0 2024-04-25T01:01:25.954 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7fc2780019f0 con 0x7fc2800be2c0 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 >> v1:172.21.15.129:6789/0 conn(0x7fc2800be2c0 legacy=0x7fc28015fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc26c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc26c004800 0x7fc26c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc26c0051c0 0x7fc26c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800c59a0 con 0x7fc26c0039b0 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c00c160 con 0x7fc26c0051c0 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc26c00c360 con 0x7fc26c004800 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc26c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286a92640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc26c0051c0 0x7fc26c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc285a90640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc26c004800 0x7fc26c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 -- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc26c004800 msgr2=0x7fc26c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc26c004800 0x7fc26c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 -- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc26c0051c0 msgr2=0x7fc26c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:25.955 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc26c0051c0 0x7fc26c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286291640 1 -- 172.21.15.129:0/3208396263 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c003520 con 0x7fc26c0039b0 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.951+0000 7fc286a92640 1 -- 172.21.15.129:0/3208396263 reap_dead start 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.952+0000 7fc286291640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc26c003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009ac0 tx=0x7fc26c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc91b02e4f7bb71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.952+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c036070 con 0x7fc26c0039b0 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.952+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27c031400 con 0x7fc26c0039b0 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.952+0000 7fc28528f640 1 -- 172.21.15.129:0/3208396263 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c009690 con 0x7fc26c0039b0 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 -- 172.21.15.129:0/3208396263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 msgr2=0x7fc26c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc26c003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009ac0 tx=0x7fc26c005830 comp rx=0 tx=0).stop 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 -- 172.21.15.129:0/3208396263 shutdown_connections 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3208396263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc26c003dd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 -- 172.21.15.129:0/3208396263 >> 172.21.15.129:0/3208396263 conn(0x7fc2800acbe0 msgr2=0x7fc2800acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 -- 172.21.15.129:0/3208396263 shutdown_connections 2024-04-25T01:01:25.956 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.953+0000 7fc2884f5640 1 -- 172.21.15.129:0/3208396263 wait complete. 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.954+0000 7fc2884f5640 1 Processor -- start 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.954+0000 7fc2884f5640 1 -- start start 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc280156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2801583c0 0x7fc280158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc280151980 0x7fc280151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800c7c70 con 0x7fc26c0039b0 2024-04-25T01:01:25.959 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc280155fd0 con 0x7fc280151980 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc2800c7df0 con 0x7fc2801583c0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc280156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc280156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41026/0 (socket says 172.21.15.129:41026) 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 -- 172.21.15.129:0/3745680300 learned_addr learned my addr 172.21.15.129:0/3745680300 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc285a90640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2801583c0 0x7fc280158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286a92640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc280151980 0x7fc280151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 -- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2801583c0 msgr2=0x7fc280158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2801583c0 0x7fc280158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 -- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc280151980 msgr2=0x7fc280151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc280151980 0x7fc280151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 -- 172.21.15.129:0/3745680300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2801eac80 con 0x7fc26c0039b0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc286291640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc280156730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc27c034040 tx=0x7fc27c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c03f020 con 0x7fc26c0039b0 2024-04-25T01:01:25.960 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27c037040 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.955+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2801eaef0 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.956+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c036070 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.956+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2801eb4b0 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.956+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc280075570 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.957+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc27c042020 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.957+0000 7fc276ffd640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc26406be60 0x7fc26406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.958+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc27c0aa900 con 0x7fc26c0039b0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.958+0000 7fc285a90640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc26406be60 0x7fc26406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:25.961 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.958+0000 7fc285a90640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc26406be60 0x7fc26406e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc28014cd80 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:25.962 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:25.962+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc27c049030 con 0x7fc26c0039b0 2024-04-25T01:01:26.135 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.134+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc28006dcb0 con 0x7fc26c0039b0 2024-04-25T01:01:26.136 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.135+0000 7fc276ffd640 1 -- 172.21.15.129:0/3745680300 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12932 (secure 0 0 0) 0x7fc27c078fb0 con 0x7fc26c0039b0 2024-04-25T01:01:26.136 INFO:teuthology.misc.health.smithi129.stdout: 2024-04-25T01:01:26.136 INFO:teuthology.misc.health.smithi129.stdout:{"epoch":17,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:23.334030+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:26.137 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.136+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc26406be60 msgr2=0x7fc26406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.137 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.136+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc26406be60 0x7fc26406e320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc28014cd80 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-04-25T01:01:26.137 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 msgr2=0x7fc280156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.137 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc280156730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc27c034040 tx=0x7fc27c004730 comp rx=0 tx=0).stop 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 shutdown_connections 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc26406be60 0x7fc26406e320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2801583c0 0x7fc280158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc280151980 0x7fc280151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 --2- 172.21.15.129:0/3745680300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc26c0039b0 0x7fc280156730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 >> 172.21.15.129:0/3745680300 conn(0x7fc2800acbe0 msgr2=0x7fc2800aafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 shutdown_connections 2024-04-25T01:01:26.138 INFO:teuthology.misc.health.smithi129.stderr:2024-04-25T01:01:26.137+0000 7fc2884f5640 1 -- 172.21.15.129:0/3745680300 wait complete. 2024-04-25T01:01:26.147 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T01:01:26.148 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T01:01:26.148 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T01:01:26.148 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i a 2024-04-25T01:01:26.150 INFO:tasks.ceph.mds.a:Started 2024-04-25T01:01:26.150 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T01:01:26.151 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i c 2024-04-25T01:01:26.152 INFO:tasks.ceph.mds.c:Started 2024-04-25T01:01:26.153 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T01:01:26.153 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i b 2024-04-25T01:01:26.155 INFO:tasks.ceph.mds.b:Started 2024-04-25T01:01:26.156 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T01:01:26.156 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mds -f --cluster ceph -i d 2024-04-25T01:01:26.158 INFO:tasks.ceph.mds.d:Started 2024-04-25T01:01:26.158 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T01:01:26.158 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:26.223 INFO:tasks.ceph.mds.a.smithi129.stdout:starting mds.a at 2024-04-25T01:01:26.223 INFO:tasks.ceph.mds.c.smithi129.stdout:starting mds.c at 2024-04-25T01:01:26.238 INFO:tasks.ceph.mds.d.smithi171.stdout:starting mds.d at 2024-04-25T01:01:26.239 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-04-25T01:01:26.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.259+0000 7f8e35d22640 1 Processor -- start 2024-04-25T01:01:26.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.259+0000 7f8e35d22640 1 -- start start 2024-04-25T01:01:26.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e35d22640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8e300c4940 0x7f8e300a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e35d22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e300aab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e35d22640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e300c4d20 con 0x7f8e300c33b0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e35d22640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f8e3006fa70 con 0x7f8e300c4940 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e35d22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f8e30070bc0 con 0x7f8e300aa6d0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e2ffff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f8e300c33b0 0x7f8e300c3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55512/0 (socket says 172.21.15.129:55512) 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.260+0000 7f8e2ffff640 1 -- 172.21.15.129:0/879334893 learned_addr learned my addr 172.21.15.129:0/879334893 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2effd640 1 --2- 172.21.15.129:0/879334893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e300aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2f7fe640 1 --2- 172.21.15.129:0/879334893 >> v2:172.21.15.171:3300/0 conn(0x7f8e300c4940 0x7f8e300a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2e7fc640 1 -- 172.21.15.129:0/879334893 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855122175 0 0) 0x7f8e300c4d20 con 0x7f8e300c33b0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2e7fc640 1 -- 172.21.15.129:0/879334893 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e1c003520 con 0x7f8e300c33b0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2e7fc640 1 -- 172.21.15.129:0/879334893 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4021361983 0 0) 0x7f8e1c003520 con 0x7f8e300c33b0 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2effd640 1 -- 172.21.15.129:0/879334893 >> v2:172.21.15.171:3300/0 conn(0x7f8e300c4940 msgr2=0x7f8e300a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2effd640 1 --2- 172.21.15.129:0/879334893 >> v2:172.21.15.171:3300/0 conn(0x7f8e300c4940 0x7f8e300a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2effd640 1 -- 172.21.15.129:0/879334893 >> v1:172.21.15.129:6789/0 conn(0x7f8e300c33b0 legacy=0x7f8e300c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.261+0000 7f8e2effd640 1 -- 172.21.15.129:0/879334893 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e300c2d30 con 0x7f8e300aa6d0 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e2effd640 1 --2- 172.21.15.129:0/879334893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e300aab30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e180097a0 tx=0x7f8e1802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4b6e4adb41b730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e2e7fc640 1 -- 172.21.15.129:0/879334893 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18037070 con 0x7f8e300aa6d0 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e2e7fc640 1 -- 172.21.15.129:0/879334893 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1802ab10 con 0x7f8e300aa6d0 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e2e7fc640 1 -- 172.21.15.129:0/879334893 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280019f0 con 0x7f8e300aa6d0 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 -- 172.21.15.129:0/879334893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 msgr2=0x7f8e300aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 --2- 172.21.15.129:0/879334893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e300aab30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e180097a0 tx=0x7f8e1802a080 comp rx=0 tx=0).stop 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 -- 172.21.15.129:0/879334893 shutdown_connections 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 --2- 172.21.15.129:0/879334893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e300aab30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 --2- 172.21.15.129:0/879334893 >> v2:172.21.15.171:3300/0 conn(0x7f8e300c4940 0x7f8e300a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 -- 172.21.15.129:0/879334893 >> 172.21.15.129:0/879334893 conn(0x7f8e30066ea0 msgr2=0x7f8e300672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:26.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 -- 172.21.15.129:0/879334893 shutdown_connections 2024-04-25T01:01:26.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.262+0000 7f8e35d22640 1 -- 172.21.15.129:0/879334893 wait complete. 2024-04-25T01:01:26.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.263+0000 7f8e35d22640 1 Processor -- start 2024-04-25T01:01:26.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 -- start start 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e3009ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e300c33b0 0x7f8e3009c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 0x7f8e3009b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3006ff00 con 0x7f8e300c4940 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30071170 con 0x7f8e300c33b0 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e35d22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f8e300c2170 con 0x7f8e300aa6d0 2024-04-25T01:01:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 0x7f8e3009b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 0x7f8e3009b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41096/0 (socket says 172.21.15.129:41096) 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e2ffff640 1 -- 172.21.15.129:0/3980066343 learned_addr learned my addr 172.21.15.129:0/3980066343 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e2f7fe640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e3009ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.264+0000 7f8e2effd640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e300c33b0 0x7f8e3009c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2ffff640 1 -- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 msgr2=0x7f8e3009ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2ffff640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e3009ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2ffff640 1 -- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e300c33b0 msgr2=0x7f8e3009c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2ffff640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e300c33b0 0x7f8e3009c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2ffff640 1 -- 172.21.15.129:0/3980066343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e301ebe00 con 0x7f8e300c4940 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2effd640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e300c33b0 0x7f8e3009c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e2ffff640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 0x7f8e3009b710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800cc90 tx=0x7f8e2800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28017070 con 0x7f8e300c4940 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e28012400 con 0x7f8e300c4940 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280127b0 con 0x7f8e300c4940 2024-04-25T01:01:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3009ac30 con 0x7f8e300c4940 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.265+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3009bc50 con 0x7f8e300c4940 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.266+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3009aa20 con 0x7f8e300c4940 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.266+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e2801a070 con 0x7f8e300c4940 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.267+0000 7f8e27fff640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f8e1006be30 0x7f8e1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.267+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f8e2808a950 con 0x7f8e300c4940 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.267+0000 7f8e2f7fe640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f8e1006be30 0x7f8e1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.267+0000 7f8e2f7fe640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f8e1006be30 0x7f8e1006e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e30156460 tx=0x7f8e20009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.271+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e28059090 con 0x7f8e300c4940 2024-04-25T01:01:26.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.445+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e30075550 con 0x7f8e300c4940 2024-04-25T01:01:26.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.446+0000 7f8e27fff640 1 -- 172.21.15.129:0/3980066343 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12932 (secure 0 0 0) 0x7f8e2805d390 con 0x7f8e300c4940 2024-04-25T01:01:26.446 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:26.446 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":17,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:23.334030+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:26.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f8e1006be30 msgr2=0x7f8e1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f8e1006be30 0x7f8e1006e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e30156460 tx=0x7f8e20009190 comp rx=0 tx=0).stop 2024-04-25T01:01:26.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 msgr2=0x7f8e3009b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 0x7f8e3009b710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800cc90 tx=0x7f8e2800a430 comp rx=0 tx=0).stop 2024-04-25T01:01:26.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 shutdown_connections 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f8e1006be30 0x7f8e1006e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e300c4940 0x7f8e3009b710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e300c33b0 0x7f8e3009c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 --2- 172.21.15.129:0/3980066343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e300aa6d0 0x7f8e3009ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 >> 172.21.15.129:0/3980066343 conn(0x7f8e30066ea0 msgr2=0x7f8e3006c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 shutdown_connections 2024-04-25T01:01:26.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.448+0000 7f8e35d22640 1 -- 172.21.15.129:0/3980066343 wait complete. 2024-04-25T01:01:26.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:01:26.460 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:01:26.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.558+0000 7fdcc9831640 1 Processor -- start 2024-04-25T01:01:26.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.558+0000 7fdcc9831640 1 -- start start 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc9831640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcc4152c70 0x7fdcc40c2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc9831640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdcc40c5810 0x7fdcc40a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc9831640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c0d40 0x7fdcc40c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc9831640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc406c000 con 0x7fdcc4152c70 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc9831640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc41560e0 con 0x7fdcc40c5810 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc9831640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc40c2310 con 0x7fdcc40c0d40 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcc4152c70 0x7fdcc40c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc2ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcc4152c70 0x7fdcc40c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41098/0 (socket says 172.21.15.129:41098) 2024-04-25T01:01:26.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc2ffd640 1 -- 172.21.15.129:0/2050693664 learned_addr learned my addr 172.21.15.129:0/2050693664 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcbbfff640 1 --2- 172.21.15.129:0/2050693664 >> v2:172.21.15.171:3300/0 conn(0x7fdcc40c5810 0x7fdcc40a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.559+0000 7fdcc37fe640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c0d40 0x7fdcc40c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc2ffd640 1 -- 172.21.15.129:0/2050693664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c0d40 msgr2=0x7fdcc40c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc2ffd640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c0d40 0x7fdcc40c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc2ffd640 1 -- 172.21.15.129:0/2050693664 >> v2:172.21.15.171:3300/0 conn(0x7fdcc40c5810 msgr2=0x7fdcc40a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc2ffd640 1 --2- 172.21.15.129:0/2050693664 >> v2:172.21.15.171:3300/0 conn(0x7fdcc40c5810 0x7fdcc40a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc2ffd640 1 -- 172.21.15.129:0/2050693664 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc406bd20 con 0x7fdcc4152c70 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc37fe640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c0d40 0x7fdcc40c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcbbfff640 1 --2- 172.21.15.129:0/2050693664 >> v2:172.21.15.171:3300/0 conn(0x7fdcc40c5810 0x7fdcc40a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc2ffd640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc4152c70 0x7fdcc40c2810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4075270 tx=0x7fdcb402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2e3e2d0332c675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc27fc640 1 -- 172.21.15.129:0/2050693664 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4037070 con 0x7fdcc4152c70 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc27fc640 1 -- 172.21.15.129:0/2050693664 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb4032440 con 0x7fdcc4152c70 2024-04-25T01:01:26.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.560+0000 7fdcc27fc640 1 -- 172.21.15.129:0/2050693664 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4032810 con 0x7fdcc4152c70 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 -- 172.21.15.129:0/2050693664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc4152c70 msgr2=0x7fdcc40c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc4152c70 0x7fdcc40c2810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4075270 tx=0x7fdcb402a140 comp rx=0 tx=0).stop 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 -- 172.21.15.129:0/2050693664 shutdown_connections 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c0d40 0x7fdcc40c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 --2- 172.21.15.129:0/2050693664 >> v2:172.21.15.171:3300/0 conn(0x7fdcc40c5810 0x7fdcc40a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 --2- 172.21.15.129:0/2050693664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc4152c70 0x7fdcc40c2810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 -- 172.21.15.129:0/2050693664 >> 172.21.15.129:0/2050693664 conn(0x7fdcc40a96c0 msgr2=0x7fdcc40a9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 -- 172.21.15.129:0/2050693664 shutdown_connections 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.561+0000 7fdcc9831640 1 -- 172.21.15.129:0/2050693664 wait complete. 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.562+0000 7fdcc9831640 1 Processor -- start 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.562+0000 7fdcc9831640 1 -- start start 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c0d40 0x7fdcc41ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 0x7fdcc40ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc4152c70 0x7fdcc4158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc406b070 con 0x7fdcc40c5810 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4153050 con 0x7fdcc40c0d40 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc406b1f0 con 0x7fdcc4152c70 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 0x7fdcc40ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 0x7fdcc40ab600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41104/0 (socket says 172.21.15.129:41104) 2024-04-25T01:01:26.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 -- 172.21.15.129:0/807353735 learned_addr learned my addr 172.21.15.129:0/807353735 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c0d40 0x7fdcc41ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc37fe640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc4152c70 0x7fdcc4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 -- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc4152c70 msgr2=0x7fdcc4158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc4152c70 0x7fdcc4158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 -- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c0d40 msgr2=0x7fdcc41ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c0d40 0x7fdcc41ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 -- 172.21.15.129:0/807353735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc40ac9c0 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcbbfff640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 0x7fdcc40ab600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e760 tx=0x7fdcac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00f040 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcac004230 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41eea60 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.563+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00ee70 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.564+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41eed40 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.564+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc4075550 con 0x7fdcc40c5810 2024-04-25T01:01:26.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.565+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcac00c6d0 con 0x7fdcc40c5810 2024-04-25T01:01:26.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.565+0000 7fdcc0ff9640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca406be30 0x7fdca406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.565+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fdcac08a740 con 0x7fdcc40c5810 2024-04-25T01:01:26.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.566+0000 7fdcc2ffd640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca406be30 0x7fdca406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.566+0000 7fdcc2ffd640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca406be30 0x7fdca406e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4034000 tx=0x7fdcb402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.570+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdcac017020 con 0x7fdcc40c5810 2024-04-25T01:01:26.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.741+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcc4074aa0 con 0x7fdcc40c5810 2024-04-25T01:01:26.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.741+0000 7fdcc0ff9640 1 -- 172.21.15.129:0/807353735 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fdcac058df0 con 0x7fdcc40c5810 2024-04-25T01:01:26.743 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:01:26.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.743+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca406be30 msgr2=0x7fdca406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.743+0000 7fdcc9831640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca406be30 0x7fdca406e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4034000 tx=0x7fdcb402e000 comp rx=0 tx=0).stop 2024-04-25T01:01:26.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.743+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 msgr2=0x7fdcc40ab600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.743+0000 7fdcc9831640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 0x7fdcc40ab600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e760 tx=0x7fdcac00ec20 comp rx=0 tx=0).stop 2024-04-25T01:01:26.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 shutdown_connections 2024-04-25T01:01:26.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca406be30 0x7fdca406e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc4152c70 0x7fdcc4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c5810 0x7fdcc40ab600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 --2- 172.21.15.129:0/807353735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c0d40 0x7fdcc41ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 >> 172.21.15.129:0/807353735 conn(0x7fdcc40a96c0 msgr2=0x7fdcc4071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:26.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 shutdown_connections 2024-04-25T01:01:26.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.744+0000 7fdcc9831640 1 -- 172.21.15.129:0/807353735 wait complete. 2024-04-25T01:01:26.755 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:01:26.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.852+0000 7fcce8438640 1 Processor -- start 2024-04-25T01:01:26.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.852+0000 7fcce8438640 1 -- start start 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce8438640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcce00c6110 0x7fcce0153f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce8438640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce00c8070 con 0x7fcce006ef80 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce8438640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce00c59d0 con 0x7fcce00a6ec0 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce8438640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fcce0156930 con 0x7fcce00c6110 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce69d5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fcce00c6110 0x7fcce0153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce61d4640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fcce006ef80 0x7fcce0153810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55524/0 (socket says 172.21.15.129:55524) 2024-04-25T01:01:26.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce61d4640 1 -- 172.21.15.129:0/1381549557 learned_addr learned my addr 172.21.15.129:0/1381549557 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:26.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce69d5640 1 -- 172.21.15.129:0/1381549557 >> v1:172.21.15.171:6789/0 conn(0x7fcce00a6ec0 legacy=0x7fcce00c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:26.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce69d5640 1 -- 172.21.15.129:0/1381549557 >> v1:172.21.15.129:6789/0 conn(0x7fcce006ef80 legacy=0x7fcce0153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce51d2640 1 -- 172.21.15.129:0/1381549557 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330758437 0 0) 0x7fcce00c8070 con 0x7fcce006ef80 2024-04-25T01:01:26.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.853+0000 7fcce69d5640 1 -- 172.21.15.129:0/1381549557 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce00c5cb0 con 0x7fcce00c6110 2024-04-25T01:01:26.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.854+0000 7fcce69d5640 1 --2- 172.21.15.129:0/1381549557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce00c6110 0x7fcce0153f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009d90 tx=0x7fccd802a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2080059067daf042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.854+0000 7fcce51d2640 1 -- 172.21.15.129:0/1381549557 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8007740 con 0x7fcce00c6110 2024-04-25T01:01:26.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.854+0000 7fcce51d2640 1 -- 172.21.15.129:0/1381549557 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd80078e0 con 0x7fcce00c6110 2024-04-25T01:01:26.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.854+0000 7fcce51d2640 1 -- 172.21.15.129:0/1381549557 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8007cb0 con 0x7fcce00c6110 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 -- 172.21.15.129:0/1381549557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce00c6110 msgr2=0x7fcce0153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 --2- 172.21.15.129:0/1381549557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce00c6110 0x7fcce0153f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009d90 tx=0x7fccd802a580 comp rx=0 tx=0).stop 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 -- 172.21.15.129:0/1381549557 shutdown_connections 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 --2- 172.21.15.129:0/1381549557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce00c6110 0x7fcce0153f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 -- 172.21.15.129:0/1381549557 >> 172.21.15.129:0/1381549557 conn(0x7fcce00a9730 msgr2=0x7fcce00a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 -- 172.21.15.129:0/1381549557 shutdown_connections 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.855+0000 7fcce8438640 1 -- 172.21.15.129:0/1381549557 wait complete. 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.856+0000 7fcce8438640 1 Processor -- start 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.856+0000 7fcce8438640 1 -- start start 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 0x7fcce0154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcce00a6ec0 0x7fcce01e1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcce00c6110 0x7fcce01ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0150c90 con 0x7fcce006ef80 2024-04-25T01:01:26.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00c8350 con 0x7fcce00a6ec0 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fcce0150e10 con 0x7fcce00c6110 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 0x7fcce0154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 0x7fcce0154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41126/0 (socket says 172.21.15.129:41126) 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 -- 172.21.15.129:0/3364926308 learned_addr learned my addr 172.21.15.129:0/3364926308 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce69d5640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcce00c6110 0x7fcce01ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce59d3640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcce00a6ec0 0x7fcce01e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 -- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcce00c6110 msgr2=0x7fcce01ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcce00c6110 0x7fcce01ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 -- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcce00a6ec0 msgr2=0x7fcce01e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcce00a6ec0 0x7fcce01e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 -- 172.21.15.129:0/3364926308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce00ac400 con 0x7fcce006ef80 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce61d4640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 0x7fcce0154ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00c500 tx=0x7fccdc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc018070 con 0x7fcce006ef80 2024-04-25T01:01:26.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc007580 con 0x7fcce006ef80 2024-04-25T01:01:26.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce0152e90 con 0x7fcce006ef80 2024-04-25T01:01:26.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc004a20 con 0x7fcce006ef80 2024-04-25T01:01:26.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.857+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce00ac6c0 con 0x7fcce006ef80 2024-04-25T01:01:26.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.858+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce0075550 con 0x7fcce006ef80 2024-04-25T01:01:26.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.859+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccdc01b070 con 0x7fcce006ef80 2024-04-25T01:01:26.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.859+0000 7fccd6ffd640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fccc406be60 0x7fccc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:26.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.859+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fccdc089d70 con 0x7fcce006ef80 2024-04-25T01:01:26.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.859+0000 7fcce59d3640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fccc406be60 0x7fccc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:26.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.860+0000 7fcce59d3640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fccc406be60 0x7fccc406e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcce0155820 tx=0x7fccd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:26.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:26.863+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fccdc058b00 con 0x7fcce006ef80 2024-04-25T01:01:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.071+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcce01530b0 con 0x7fcce006ef80 2024-04-25T01:01:27.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.072+0000 7fccd6ffd640 1 -- 172.21.15.129:0/3364926308 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fccdc058400 con 0x7fcce006ef80 2024-04-25T01:01:27.073 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:27.073 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:01:27.073 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 2 2024-04-25T01:01:27.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fccc406be60 msgr2=0x7fccc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:27.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fccc406be60 0x7fccc406e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcce0155820 tx=0x7fccd0009040 comp rx=0 tx=0).stop 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 msgr2=0x7fcce0154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 0x7fcce0154ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00c500 tx=0x7fccdc00c9c0 comp rx=0 tx=0).stop 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 shutdown_connections 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fccc406be60 0x7fccc406e320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcce00c6110 0x7fcce01ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcce00a6ec0 0x7fcce01e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 --2- 172.21.15.129:0/3364926308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcce006ef80 0x7fcce0154ff0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 >> 172.21.15.129:0/3364926308 conn(0x7fcce00a9730 msgr2=0x7fcce0155f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 shutdown_connections 2024-04-25T01:01:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:27.074+0000 7fcce8438640 1 -- 172.21.15.129:0/3364926308 wait complete. 2024-04-25T01:01:27.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:01:27.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:01:29.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:01:29.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.194+0000 7ff256710640 1 Processor -- start 2024-04-25T01:01:29.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.194+0000 7ff256710640 1 -- start start 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff256710640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 0x7ff25014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff256710640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2500c4f10 con 0x7ff2500ac540 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff256710640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2500c5180 con 0x7ff2500abe50 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff256710640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff2500ac2a0 con 0x7ff25014f6f0 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24f7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff2500ac540 0x7ff2500c4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:51908/0 (socket says 172.21.15.129:51908) 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24f7fe640 1 -- 172.21.15.129:0/3708849727 learned_addr learned my addr 172.21.15.129:0/3708849727 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 0x7ff25014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294706107 0 0) 0x7ff2500c4f10 con 0x7ff2500ac540 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff238003520 con 0x7ff2500ac540 2024-04-25T01:01:29.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 983766359 0 0) 0x7ff238003520 con 0x7ff2500ac540 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 msgr2=0x7ff25014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 0x7ff25014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 >> v1:172.21.15.171:6789/0 conn(0x7ff2500abe50 legacy=0x7ff25006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff250155720 con 0x7ff2500ac540 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7ff240001670 con 0x7ff2500ac540 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 >> v1:172.21.15.129:6789/0 conn(0x7ff2500ac540 legacy=0x7ff2500c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2380039e0 0x7ff238003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 0x7ff238004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238004d40 0x7ff2380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2500ac2a0 con 0x7ff2380039e0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff23800ba70 con 0x7ff238004d40 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.195+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff23800bc70 con 0x7ff25014f6f0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2380039e0 0x7ff238003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24f7fe640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 0x7ff238004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff254cad640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238004d40 0x7ff2380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 -- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 msgr2=0x7ff238004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff25014f6f0 0x7ff238004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 -- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238004d40 msgr2=0x7ff2380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff238004d40 0x7ff2380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 -- 172.21.15.129:0/3708849727 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff238003520 con 0x7ff2380039e0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.196+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2380039e0 0x7ff238003dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2440093c0 tx=0x7ff238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6217c649055858f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244038070 con 0x7ff2380039e0 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff244033400 con 0x7ff2380039e0 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff24effd640 1 -- 172.21.15.129:0/3708849727 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244009690 con 0x7ff2380039e0 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff256710640 1 -- 172.21.15.129:0/3708849727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2380039e0 msgr2=0x7ff238003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff256710640 1 --2- 172.21.15.129:0/3708849727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2380039e0 0x7ff238003dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2440093c0 tx=0x7ff238005830 comp rx=0 tx=0).stop 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff254cad640 1 -- 172.21.15.129:0/3708849727 reap_dead start 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff256710640 1 -- 172.21.15.129:0/3708849727 shutdown_connections 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff256710640 1 -- 172.21.15.129:0/3708849727 >> 172.21.15.129:0/3708849727 conn(0x7ff2500a86b0 msgr2=0x7ff2500a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff256710640 1 -- 172.21.15.129:0/3708849727 shutdown_connections 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.197+0000 7ff256710640 1 -- 172.21.15.129:0/3708849727 wait complete. 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.198+0000 7ff256710640 1 Processor -- start 2024-04-25T01:01:29.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.198+0000 7ff256710640 1 -- start start 2024-04-25T01:01:29.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff256710640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff2380039e0 0x7ff250152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff256710640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2500c0ac0 0x7ff2500c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff256710640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 0x7ff2501e6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff256710640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff250155cc0 con 0x7ff2500c2a60 2024-04-25T01:01:29.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff256710640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501559e0 con 0x7ff2500c0ac0 2024-04-25T01:01:29.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff256710640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff250155fa0 con 0x7ff2380039e0 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 0x7ff2501e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 0x7ff2501e6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39390/0 (socket says 172.21.15.129:39390) 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 -- 172.21.15.129:0/3979525304 learned_addr learned my addr 172.21.15.129:0/3979525304 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff24f7fe640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2500c0ac0 0x7ff2500c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff2380039e0 0x7ff250152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 -- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff2380039e0 msgr2=0x7ff250152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff2380039e0 0x7ff250152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 -- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2500c0ac0 msgr2=0x7ff2500c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.199+0000 7ff254cad640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2500c0ac0 0x7ff2500c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff254cad640 1 -- 172.21.15.129:0/3979525304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2501ea2e0 con 0x7ff2500c2a60 2024-04-25T01:01:29.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff254cad640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 0x7ff2501e6330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2500aa070 tx=0x7ff24800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248019070 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff248006d60 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2500725a0 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248003bb0 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.200+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff250072b60 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.201+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff250075550 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.201+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff24801c070 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.201+0000 7ff24cff9640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff23c06be30 0x7ff23c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.202+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff248089e90 con 0x7ff2500c2a60 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.202+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff23c06be30 0x7ff23c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.202+0000 7ff24ffff640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff23c06be30 0x7ff23c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff244035000 tx=0x7ff244008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.206+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff248058c20 con 0x7ff2500c2a60 2024-04-25T01:01:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.419+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff25006e600 con 0x7ff2500c2a60 2024-04-25T01:01:29.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.419+0000 7ff24cff9640 1 -- 172.21.15.129:0/3979525304 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7ff248058520 con 0x7ff2500c2a60 2024-04-25T01:01:29.421 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:29.421 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:01:29.422 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 3 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff23c06be30 msgr2=0x7ff23c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff23c06be30 0x7ff23c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff244035000 tx=0x7ff244008030 comp rx=0 tx=0).stop 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 msgr2=0x7ff2501e6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 0x7ff2501e6330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2500aa070 tx=0x7ff24800ff70 comp rx=0 tx=0).stop 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 shutdown_connections 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff23c06be30 0x7ff23c06e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2500c2a60 0x7ff2501e6330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2500c0ac0 0x7ff2500c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 --2- 172.21.15.129:0/3979525304 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff2380039e0 0x7ff250152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 >> 172.21.15.129:0/3979525304 conn(0x7ff2500a86b0 msgr2=0x7ff250153520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 shutdown_connections 2024-04-25T01:01:29.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.422+0000 7ff256710640 1 -- 172.21.15.129:0/3979525304 wait complete. 2024-04-25T01:01:29.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:29.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.530+0000 7f0c4445d640 1 Processor -- start 2024-04-25T01:01:29.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.530+0000 7f0c4445d640 1 -- start start 2024-04-25T01:01:29.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c4445d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0c3c14f260 0x7f0c3c0c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c4445d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0c3c155d70 0x7f0c3c0c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c4445d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c0733a0 con 0x7f0c3c067300 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c4445d640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c156180 con 0x7f0c3c155d70 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c4445d640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c158c50 con 0x7f0c3c14f260 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0c3c155d70 0x7f0c3c0c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0c3c155d70 0x7f0c3c0c1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39398/0 (socket says 172.21.15.129:39398) 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 -- 172.21.15.129:0/1406850043 learned_addr learned my addr 172.21.15.129:0/1406850043 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c421f9640 1 --2- 172.21.15.129:0/1406850043 >> v2:172.21.15.171:3300/0 conn(0x7f0c3c14f260 0x7f0c3c0c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 -- 172.21.15.129:0/1406850043 >> v2:172.21.15.171:3300/0 conn(0x7f0c3c14f260 msgr2=0x7f0c3c0c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 --2- 172.21.15.129:0/1406850043 >> v2:172.21.15.171:3300/0 conn(0x7f0c3c14f260 0x7f0c3c0c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 -- 172.21.15.129:0/1406850043 >> v1:172.21.15.171:6789/0 conn(0x7f0c3c067300 legacy=0x7f0c3c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:29.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.531+0000 7f0c429fa640 1 -- 172.21.15.129:0/1406850043 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c073680 con 0x7f0c3c155d70 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.532+0000 7f0c429fa640 1 --2- 172.21.15.129:0/1406850043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c155d70 0x7f0c3c0c1bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c06f040 tx=0x7f0c3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400df652a916b3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.532+0000 7f0c411f7640 1 -- 172.21.15.129:0/1406850043 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34037070 con 0x7f0c3c155d70 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.532+0000 7f0c411f7640 1 -- 172.21.15.129:0/1406850043 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c34032440 con 0x7f0c3c155d70 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.532+0000 7f0c411f7640 1 -- 172.21.15.129:0/1406850043 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34032810 con 0x7f0c3c155d70 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 -- 172.21.15.129:0/1406850043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c155d70 msgr2=0x7f0c3c0c1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1406850043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c155d70 0x7f0c3c0c1bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c06f040 tx=0x7f0c3402a1a0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 -- 172.21.15.129:0/1406850043 shutdown_connections 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1406850043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c155d70 0x7f0c3c0c1bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1406850043 >> v2:172.21.15.171:3300/0 conn(0x7f0c3c14f260 0x7f0c3c0c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 -- 172.21.15.129:0/1406850043 >> 172.21.15.129:0/1406850043 conn(0x7f0c3c0c71f0 msgr2=0x7f0c3c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 -- 172.21.15.129:0/1406850043 shutdown_connections 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.533+0000 7f0c4445d640 1 -- 172.21.15.129:0/1406850043 wait complete. 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.534+0000 7f0c4445d640 1 Processor -- start 2024-04-25T01:01:29.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.534+0000 7f0c4445d640 1 -- start start 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 0x7f0c3c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c3c14f260 0x7f0c3c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c3c155d70 0x7f0c3c0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c06b400 con 0x7f0c3c067300 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1597f0 con 0x7f0c3c14f260 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c14f640 con 0x7f0c3c155d70 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 0x7f0c3c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 0x7f0c3c153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39408/0 (socket says 172.21.15.129:39408) 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 -- 172.21.15.129:0/1523538309 learned_addr learned my addr 172.21.15.129:0/1523538309 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c429fa640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c3c155d70 0x7f0c3c0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c419f8640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c3c14f260 0x7f0c3c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 -- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c3c155d70 msgr2=0x7f0c3c0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c3c155d70 0x7f0c3c0c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 -- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c3c14f260 msgr2=0x7f0c3c1582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c3c14f260 0x7f0c3c1582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 -- 172.21.15.129:0/1523538309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c072f20 con 0x7f0c3c067300 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c429fa640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c3c155d70 0x7f0c3c0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c421f9640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 0x7f0c3c153160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007ef0 tx=0x7f0c3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38014620 con 0x7f0c3c067300 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c38019070 con 0x7f0c3c067300 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c154e30 con 0x7f0c3c067300 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38014a00 con 0x7f0c3c067300 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.535+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c0c67f0 con 0x7f0c3c067300 2024-04-25T01:01:29.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.536+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3c075550 con 0x7f0c3c067300 2024-04-25T01:01:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.537+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c38014ba0 con 0x7f0c3c067300 2024-04-25T01:01:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.537+0000 7f0c32ffd640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0c2006be30 0x7f0c2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.537+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f0c3808b3e0 con 0x7f0c3c067300 2024-04-25T01:01:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.537+0000 7f0c419f8640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0c2006be30 0x7f0c2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.538+0000 7f0c419f8640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0c2006be30 0x7f0c2006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c006fd0 tx=0x7f0c2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.541+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c3805a170 con 0x7f0c3c067300 2024-04-25T01:01:29.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.715+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c3c069060 con 0x7f0c3c067300 2024-04-25T01:01:29.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.716+0000 7f0c32ffd640 1 -- 172.21.15.129:0/1523538309 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12932 (secure 0 0 0) 0x7f0c38059a70 con 0x7f0c3c067300 2024-04-25T01:01:29.717 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:29.717 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":17,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:23.334030+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:29.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0c2006be30 msgr2=0x7f0c2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0c2006be30 0x7f0c2006e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c006fd0 tx=0x7f0c2c008040 comp rx=0 tx=0).stop 2024-04-25T01:01:29.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 msgr2=0x7f0c3c153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 0x7f0c3c153160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007ef0 tx=0x7f0c3800e4b0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 shutdown_connections 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0c2006be30 0x7f0c2006e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c3c155d70 0x7f0c3c0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c3c14f260 0x7f0c3c1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 --2- 172.21.15.129:0/1523538309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0c3c067300 0x7f0c3c153160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 >> 172.21.15.129:0/1523538309 conn(0x7f0c3c0c71f0 msgr2=0x7f0c3c14c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 shutdown_connections 2024-04-25T01:01:29.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.718+0000 7f0c4445d640 1 -- 172.21.15.129:0/1523538309 wait complete. 2024-04-25T01:01:29.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:01:29.730 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:01:29.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.829+0000 7f5db8a8c640 1 Processor -- start 2024-04-25T01:01:29.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.829+0000 7f5db8a8c640 1 -- start start 2024-04-25T01:01:29.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db8a8c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db4152cb0 0x7f5db40c6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db8a8c640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5db40c3930 0x7f5db40c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db8a8c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db40c6ed0 con 0x7f5db40c8480 2024-04-25T01:01:29.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db8a8c640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f5db4071310 con 0x7f5db40c3930 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db8a8c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40c3670 con 0x7f5db4152cb0 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db1d9b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f5db40c8480 0x7f5db40c1dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:51918/0 (socket says 172.21.15.129:51918) 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db1d9b640 1 -- 172.21.15.129:0/741153762 learned_addr learned my addr 172.21.15.129:0/741153762 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db259c640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db4152cb0 0x7f5db40c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db2d9d640 1 --2- 172.21.15.129:0/741153762 >> v2:172.21.15.171:3300/0 conn(0x7f5db40c3930 0x7f5db40c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db159a640 1 -- 172.21.15.129:0/741153762 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239505744 0 0) 0x7f5db40c6ed0 con 0x7f5db40c8480 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.830+0000 7f5db159a640 1 -- 172.21.15.129:0/741153762 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d9c003520 con 0x7f5db40c8480 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db2d9d640 1 -- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db4152cb0 msgr2=0x7f5db40c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db2d9d640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db4152cb0 0x7f5db40c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db2d9d640 1 -- 172.21.15.129:0/741153762 >> v1:172.21.15.129:6789/0 conn(0x7f5db40c8480 legacy=0x7f5db40c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db159a640 1 -- 172.21.15.129:0/741153762 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 930045761 0 0) 0x7f5d9c003520 con 0x7f5db40c8480 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db2d9d640 1 -- 172.21.15.129:0/741153762 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4153090 con 0x7f5db40c3930 2024-04-25T01:01:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db259c640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db4152cb0 0x7f5db40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db2d9d640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db40c3930 0x7f5db40c3e70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0097a0 tx=0x7f5dac02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d2ec6b15f47275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.831+0000 7f5db159a640 1 -- 172.21.15.129:0/741153762 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac037070 con 0x7f5db40c3930 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db159a640 1 -- 172.21.15.129:0/741153762 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac02ab10 con 0x7f5db40c3930 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db159a640 1 -- 172.21.15.129:0/741153762 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8001670 con 0x7f5db40c3930 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 -- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db40c3930 msgr2=0x7f5db40c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db40c3930 0x7f5db40c3e70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0097a0 tx=0x7f5dac02a080 comp rx=0 tx=0).stop 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 -- 172.21.15.129:0/741153762 shutdown_connections 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db40c3930 0x7f5db40c3e70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/741153762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db4152cb0 0x7f5db40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 -- 172.21.15.129:0/741153762 >> 172.21.15.129:0/741153762 conn(0x7f5db40aa740 msgr2=0x7f5db40aab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 -- 172.21.15.129:0/741153762 shutdown_connections 2024-04-25T01:01:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.832+0000 7f5db8a8c640 1 -- 172.21.15.129:0/741153762 wait complete. 2024-04-25T01:01:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.833+0000 7f5db8a8c640 1 Processor -- start 2024-04-25T01:01:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.833+0000 7f5db8a8c640 1 -- start start 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db8a8c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 0x7f5db414cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db8a8c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db40c8480 0x7f5db414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db8a8c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4152cb0 0x7f5db414af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db8a8c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40c4920 con 0x7f5db40c3930 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db8a8c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4159740 con 0x7f5db4152cb0 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db8a8c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40c4c00 con 0x7f5db40c8480 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 0x7f5db414cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 0x7f5db414cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39416/0 (socket says 172.21.15.129:39416) 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 -- 172.21.15.129:0/3384472353 learned_addr learned my addr 172.21.15.129:0/3384472353 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 -- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db40c8480 msgr2=0x7f5db414a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db40c8480 0x7f5db414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 -- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4152cb0 msgr2=0x7f5db414af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4152cb0 0x7f5db414af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.834+0000 7f5db259c640 1 -- 172.21.15.129:0/3384472353 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db41eae40 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.835+0000 7f5db259c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 0x7f5db414cfe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5da400efc0 tx=0x7f5da400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.835+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4018020 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.835+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da4014680 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.835+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db41eb120 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.835+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4014a50 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.835+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41eb6e0 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.836+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db4075550 con 0x7f5db40c3930 2024-04-25T01:01:29.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.836+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5da4005d10 con 0x7f5db40c3930 2024-04-25T01:01:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.836+0000 7f5da2ffd640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5d9406be30 0x7f5d9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.836+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5da408b8c0 con 0x7f5db40c3930 2024-04-25T01:01:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.837+0000 7f5db1d9b640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5d9406be30 0x7f5d9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:29.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.837+0000 7f5db1d9b640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5d9406be30 0x7f5d9406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5db40ad050 tx=0x7f5da8001180 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:29.841+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5da4022030 con 0x7f5db40c3930 2024-04-25T01:01:30.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.017+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5db4068630 con 0x7f5db40c3930 2024-04-25T01:01:30.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.017+0000 7f5da2ffd640 1 -- 172.21.15.129:0/3384472353 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f5da4010040 con 0x7f5db40c3930 2024-04-25T01:01:30.019 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5d9406be30 msgr2=0x7f5d9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5d9406be30 0x7f5d9406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5db40ad050 tx=0x7f5da8001180 comp rx=0 tx=0).stop 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 msgr2=0x7f5db414cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 0x7f5db414cfe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5da400efc0 tx=0x7f5da400c490 comp rx=0 tx=0).stop 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 shutdown_connections 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5d9406be30 0x7f5d9406e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5db40c8480 0x7f5db414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4152cb0 0x7f5db414af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 --2- 172.21.15.129:0/3384472353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db40c3930 0x7f5db414cfe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 >> 172.21.15.129:0/3384472353 conn(0x7f5db40aa740 msgr2=0x7f5db406e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:30.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 shutdown_connections 2024-04-25T01:01:30.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.019+0000 7f5db8a8c640 1 -- 172.21.15.129:0/3384472353 wait complete. 2024-04-25T01:01:30.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:01:30.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.127+0000 7f325cfc8640 1 Processor -- start 2024-04-25T01:01:30.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.127+0000 7f325cfc8640 1 -- start start 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f325cfc8640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3258152bc0 0x7f32580c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f325cfc8640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f32580c0470 0x7f3258159300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f325cfc8640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f325806e780 con 0x7f325806cb10 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f325cfc8640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f325806c680 con 0x7f32580c0470 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f325cfc8640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f3258159840 con 0x7f3258152bc0 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f32580c0470 0x7f3258159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f32580c0470 0x7f3258159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39426/0 (socket says 172.21.15.129:39426) 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 -- 172.21.15.129:0/3105458274 learned_addr learned my addr 172.21.15.129:0/3105458274 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f325659c640 1 --2- 172.21.15.129:0/3105458274 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3258152bc0 0x7f32580c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 -- 172.21.15.129:0/3105458274 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3258152bc0 msgr2=0x7f32580c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 --2- 172.21.15.129:0/3105458274 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3258152bc0 0x7f32580c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 -- 172.21.15.129:0/3105458274 >> v1:172.21.15.171:6789/0 conn(0x7f325806cb10 legacy=0x7f325806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.128+0000 7f324ffff640 1 -- 172.21.15.129:0/3105458274 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32580c0970 con 0x7f32580c0470 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f324ffff640 1 --2- 172.21.15.129:0/3105458274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32580c0470 0x7f3258159300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3240009870 tx=0x7f324002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacb79105dc7e130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f3255d9b640 1 -- 172.21.15.129:0/3105458274 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240037070 con 0x7f32580c0470 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f3255d9b640 1 -- 172.21.15.129:0/3105458274 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3240032440 con 0x7f32580c0470 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f3255d9b640 1 -- 172.21.15.129:0/3105458274 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240032810 con 0x7f32580c0470 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 -- 172.21.15.129:0/3105458274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32580c0470 msgr2=0x7f3258159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 --2- 172.21.15.129:0/3105458274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32580c0470 0x7f3258159300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3240009870 tx=0x7f324002a210 comp rx=0 tx=0).stop 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 -- 172.21.15.129:0/3105458274 shutdown_connections 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 --2- 172.21.15.129:0/3105458274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32580c0470 0x7f3258159300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 --2- 172.21.15.129:0/3105458274 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3258152bc0 0x7f32580c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 -- 172.21.15.129:0/3105458274 >> 172.21.15.129:0/3105458274 conn(0x7f32580a86b0 msgr2=0x7f32580a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:30.132 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 -- 172.21.15.129:0/3105458274 shutdown_connections 2024-04-25T01:01:30.133 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.129+0000 7f325cfc8640 1 -- 172.21.15.129:0/3105458274 wait complete. 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 Processor -- start 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 -- start start 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f325806cb10 0x7f32581e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f32580c0470 0x7f32581ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 0x7f32581eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32580c0f30 con 0x7f3258152bc0 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f325806c310 con 0x7f325806cb10 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f325cfc8640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f325806ddd0 con 0x7f32580c0470 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f3256d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 0x7f32581eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f3256d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 0x7f32581eed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39436/0 (socket says 172.21.15.129:39436) 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.131+0000 7f3256d9d640 1 -- 172.21.15.129:0/2957822084 learned_addr learned my addr 172.21.15.129:0/2957822084 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f3256d9d640 1 -- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f32580c0470 msgr2=0x7f32581ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f324ffff640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f32580c0470 0x7f32581ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f325659c640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f325806cb10 0x7f32581e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f3256d9d640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f32580c0470 0x7f32581ee7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f3256d9d640 1 -- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f325806cb10 msgr2=0x7f32581e0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f3256d9d640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f325806cb10 0x7f32581e0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f3256d9d640 1 -- 172.21.15.129:0/2957822084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32581df920 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f3256d9d640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 0x7f32581eed00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f325000c7a0 tx=0x7f325000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250016020 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325000a9f0 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32581dfc00 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325000ae20 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.132+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32581e1490 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.133+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3258075550 con 0x7f3258152bc0 2024-04-25T01:01:30.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.133+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3250019020 con 0x7f3258152bc0 2024-04-25T01:01:30.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.134+0000 7f324effd640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f323806be60 0x7f323806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.134+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f325008a9f0 con 0x7f3258152bc0 2024-04-25T01:01:30.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.134+0000 7f325659c640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f323806be60 0x7f323806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.134+0000 7f325659c640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f323806be60 0x7f323806e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:30.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.138+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3250020030 con 0x7f3258152bc0 2024-04-25T01:01:30.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.316+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f32580acc90 con 0x7f3258152bc0 2024-04-25T01:01:30.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.316+0000 7f324effd640 1 -- 172.21.15.129:0/2957822084 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f3250059130 con 0x7f3258152bc0 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout:[ 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout: "poolnum": 1, 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout: "poolname": ".mgr" 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T01:01:30.318 INFO:teuthology.orchestra.run.smithi129.stdout:] 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f323806be60 msgr2=0x7f323806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f323806be60 0x7f323806e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248008040 comp rx=0 tx=0).stop 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 msgr2=0x7f32581eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 0x7f32581eed00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f325000c7a0 tx=0x7f325000cc60 comp rx=0 tx=0).stop 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 shutdown_connections 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f323806be60 0x7f323806e320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f32580c0470 0x7f32581ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3258152bc0 0x7f32581eed00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 --2- 172.21.15.129:0/2957822084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f325806cb10 0x7f32581e0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 >> 172.21.15.129:0/2957822084 conn(0x7f32580a86b0 msgr2=0x7f32580c44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 shutdown_connections 2024-04-25T01:01:30.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.318+0000 7f325cfc8640 1 -- 172.21.15.129:0/2957822084 wait complete. 2024-04-25T01:01:30.330 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T01:01:30.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T01:01:30.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.431+0000 7fcb8536a640 1 Processor -- start 2024-04-25T01:01:30.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.431+0000 7fcb8536a640 1 -- start start 2024-04-25T01:01:30.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.431+0000 7fcb8536a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb80158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.431+0000 7fcb8536a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcb800c4e80 0x7fcb80155de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb8536a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb80159000 con 0x7fcb80070310 2024-04-25T01:01:30.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb8536a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fcb800c8d10 con 0x7fcb800c4e80 2024-04-25T01:01:30.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb8536a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fcb800c8fd0 con 0x7fcb80070c90 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7f7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fcb80070310 0x7fcb8014b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:51934/0 (socket says 172.21.15.129:51934) 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7f7fe640 1 -- 172.21.15.129:0/2128351880 learned_addr learned my addr 172.21.15.129:0/2128351880 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7effd640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb80158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb77fff640 1 --2- 172.21.15.129:0/2128351880 >> v2:172.21.15.171:3300/0 conn(0x7fcb800c4e80 0x7fcb80155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136017865 0 0) 0x7fcb80159000 con 0x7fcb80070310 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb6c003520 con 0x7fcb80070310 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3565357544 0 0) 0x7fcb6c003520 con 0x7fcb80070310 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 msgr2=0x7fcb80158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb80158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 >> v2:172.21.15.171:3300/0 conn(0x7fcb800c4e80 msgr2=0x7fcb80155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 --2- 172.21.15.129:0/2128351880 >> v2:172.21.15.171:3300/0 conn(0x7fcb800c4e80 0x7fcb80155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb800c92b0 con 0x7fcb80070310 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7fcb780019f0 con 0x7fcb80070310 2024-04-25T01:01:30.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 >> v1:172.21.15.129:6789/0 conn(0x7fcb80070310 legacy=0x7fcb8014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb6c004800 0x7fcb6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb800c8d10 con 0x7fcb6c0039b0 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb800c8fd0 con 0x7fcb6c004800 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6c00b9c0 con 0x7fcb80070c90 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7effd640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb7f7fe640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.432+0000 7fcb77fff640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb6c004800 0x7fcb6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7effd640 1 -- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 msgr2=0x7fcb6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7effd640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7effd640 1 -- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb6c004800 msgr2=0x7fcb6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7effd640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb6c004800 0x7fcb6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7effd640 1 -- 172.21.15.129:0/2128351880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c003520 con 0x7fcb6c0039b0 2024-04-25T01:01:30.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7f7fe640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb80070c90 0x7fcb6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7effd640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb6c003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009ac0 tx=0x7fcb6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561c5df8684ab242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70036070 con 0x7fcb6c0039b0 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70031400 con 0x7fcb6c0039b0 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.433+0000 7fcb7e7fc640 1 -- 172.21.15.129:0/2128351880 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70009690 con 0x7fcb6c0039b0 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb8536a640 1 -- 172.21.15.129:0/2128351880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 msgr2=0x7fcb6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb8536a640 1 --2- 172.21.15.129:0/2128351880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb6c003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009ac0 tx=0x7fcb6c005830 comp rx=0 tx=0).stop 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb7f7fe640 1 -- 172.21.15.129:0/2128351880 reap_dead start 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb8536a640 1 -- 172.21.15.129:0/2128351880 shutdown_connections 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb8536a640 1 -- 172.21.15.129:0/2128351880 >> 172.21.15.129:0/2128351880 conn(0x7fcb80066ea0 msgr2=0x7fcb80067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb8536a640 1 -- 172.21.15.129:0/2128351880 shutdown_connections 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.434+0000 7fcb8536a640 1 -- 172.21.15.129:0/2128351880 wait complete. 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.435+0000 7fcb8536a640 1 Processor -- start 2024-04-25T01:01:30.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.435+0000 7fcb8536a640 1 -- start start 2024-04-25T01:01:30.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb800beed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb80147380 0x7fcb801477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb801ebf40 0x7fcb801ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80156320 con 0x7fcb6c0039b0 2024-04-25T01:01:30.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801558c0 con 0x7fcb80147380 2024-04-25T01:01:30.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fcb800c8960 con 0x7fcb801ebf40 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb800beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb800beed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39454/0 (socket says 172.21.15.129:39454) 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 -- 172.21.15.129:0/3217301523 learned_addr learned my addr 172.21.15.129:0/3217301523 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb77fff640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb80147380 0x7fcb801477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7f7fe640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb801ebf40 0x7fcb801ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 -- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb801ebf40 msgr2=0x7fcb801ec3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb801ebf40 0x7fcb801ec3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 -- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb80147380 msgr2=0x7fcb801477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb80147380 0x7fcb801477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 -- 172.21.15.129:0/3217301523 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb801511b0 con 0x7fcb6c0039b0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb7effd640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb800beed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009ac0 tx=0x7fcb70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70040020 con 0x7fcb6c0039b0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb70003bc0 con 0x7fcb6c0039b0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb70036040 con 0x7fcb6c0039b0 2024-04-25T01:01:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb80151430 con 0x7fcb6c0039b0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.436+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb80151ac0 con 0x7fcb6c0039b0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.437+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb80068fd0 con 0x7fcb6c0039b0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.438+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb70043020 con 0x7fcb6c0039b0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.438+0000 7fcb777fe640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fcb6006be60 0x7fcb6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.438+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fcb700aaa10 con 0x7fcb6c0039b0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.438+0000 7fcb77fff640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fcb6006be60 0x7fcb6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.439+0000 7fcb77fff640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fcb6006be60 0x7fcb6006e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb800bf7c0 tx=0x7fcb68009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:30.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.442+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcb7004a070 con 0x7fcb6c0039b0 2024-04-25T01:01:30.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:30.620+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcb800700f0 con 0x7fcb6c0039b0 2024-04-25T01:01:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.372+0000 7fcb777fe640 1 -- 172.21.15.129:0/3217301523 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fcb801511b0 con 0x7fcb6c0039b0 2024-04-25T01:01:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' created 2024-04-25T01:01:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fcb6006be60 msgr2=0x7fcb6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fcb6006be60 0x7fcb6006e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb800bf7c0 tx=0x7fcb68009190 comp rx=0 tx=0).stop 2024-04-25T01:01:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 msgr2=0x7fcb800beed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb800beed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009ac0 tx=0x7fcb70004730 comp rx=0 tx=0).stop 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 shutdown_connections 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fcb6006be60 0x7fcb6006e320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb801ebf40 0x7fcb801ec3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb80147380 0x7fcb801477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 --2- 172.21.15.129:0/3217301523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb6c0039b0 0x7fcb800beed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.374+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 >> 172.21.15.129:0/3217301523 conn(0x7fcb80066ea0 msgr2=0x7fcb80153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.375+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 shutdown_connections 2024-04-25T01:01:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.375+0000 7fcb8536a640 1 -- 172.21.15.129:0/3217301523 wait complete. 2024-04-25T01:01:31.393 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T01:01:31.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.501+0000 7fc819b90640 1 Processor -- start 2024-04-25T01:01:31.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.501+0000 7fc819b90640 1 -- start start 2024-04-25T01:01:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc819b90640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc81414f730 0x7fc81414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:31.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc819b90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc8140c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:31.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc819b90640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8140c5050 con 0x7fc81406c4d0 2024-04-25T01:01:31.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc819b90640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fc8140c5330 con 0x7fc81414f730 2024-04-25T01:01:31.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc819b90640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc8140c5610 con 0x7fc814155760 2024-04-25T01:01:31.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc812ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc8140c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:31.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc8137fe640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc81414f730 0x7fc81414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc812ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc8140c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:53560/0 (socket says 172.21.15.129:53560) 2024-04-25T01:01:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc812ffd640 1 -- 172.21.15.129:0/2358869205 learned_addr learned my addr 172.21.15.129:0/2358869205 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc8137fe640 1 -- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 msgr2=0x7fc8140c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc8137fe640 1 --2- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc8140c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc8137fe640 1 -- 172.21.15.129:0/2358869205 >> v1:172.21.15.171:6789/0 conn(0x7fc81406c4d0 legacy=0x7fc8140ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc8137fe640 1 -- 172.21.15.129:0/2358869205 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8140c58f0 con 0x7fc81414f730 2024-04-25T01:01:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc8127fc640 1 -- 172.21.15.129:0/2358869205 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187864978 0 0) 0x7fc8140c5050 con 0x7fc81406c4d0 2024-04-25T01:01:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.502+0000 7fc812ffd640 1 --2- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc8140c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.503+0000 7fc8137fe640 1 --2- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc81414fb10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8040097a0 tx=0x7fc80402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bacb667c143fe0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.503+0000 7fc8127fc640 1 -- 172.21.15.129:0/2358869205 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804037070 con 0x7fc81414f730 2024-04-25T01:01:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.503+0000 7fc8127fc640 1 -- 172.21.15.129:0/2358869205 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804032440 con 0x7fc81414f730 2024-04-25T01:01:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.503+0000 7fc8127fc640 1 -- 172.21.15.129:0/2358869205 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804032810 con 0x7fc81414f730 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 -- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 msgr2=0x7fc81414fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 --2- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc81414fb10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8040097a0 tx=0x7fc80402a180 comp rx=0 tx=0).stop 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 -- 172.21.15.129:0/2358869205 shutdown_connections 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 --2- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc8140c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 --2- 172.21.15.129:0/2358869205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc81414fb10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 -- 172.21.15.129:0/2358869205 >> 172.21.15.129:0/2358869205 conn(0x7fc8140a86b0 msgr2=0x7fc8140a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 -- 172.21.15.129:0/2358869205 shutdown_connections 2024-04-25T01:01:31.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.504+0000 7fc819b90640 1 -- 172.21.15.129:0/2358869205 wait complete. 2024-04-25T01:01:31.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.505+0000 7fc819b90640 1 Processor -- start 2024-04-25T01:01:31.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.505+0000 7fc819b90640 1 -- start start 2024-04-25T01:01:31.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 0x7fc814072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:31.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc814071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:31.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc814071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:31.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc81406c900 con 0x7fc81406c4d0 2024-04-25T01:01:31.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc8140aadd0 con 0x7fc81414f730 2024-04-25T01:01:31.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc81406ca80 con 0x7fc814155760 2024-04-25T01:01:31.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 0x7fc814072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:31.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 0x7fc814072970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39456/0 (socket says 172.21.15.129:39456) 2024-04-25T01:01:31.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 -- 172.21.15.129:0/2370636552 learned_addr learned my addr 172.21.15.129:0/2370636552 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:31.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc812ffd640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc814071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:31.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc813fff640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc814071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:31.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 -- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 msgr2=0x7fc814071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:31.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc814071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 -- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 msgr2=0x7fc814071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:31.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc814071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:31.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 -- 172.21.15.129:0/2370636552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8141eb9d0 con 0x7fc81406c4d0 2024-04-25T01:01:31.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc8137fe640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 0x7fc814072970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc804035040 tx=0x7fc8040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:31.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804037050 con 0x7fc81406c4d0 2024-04-25T01:01:31.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804038040 con 0x7fc81406c4d0 2024-04-25T01:01:31.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8141ebcb0 con 0x7fc81406c4d0 2024-04-25T01:01:31.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.506+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804004490 con 0x7fc81406c4d0 2024-04-25T01:01:31.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.507+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc814071280 con 0x7fc81406c4d0 2024-04-25T01:01:31.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.507+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8140c5050 con 0x7fc81406c4d0 2024-04-25T01:01:31.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.508+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc80403c6d0 con 0x7fc81406c4d0 2024-04-25T01:01:31.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.508+0000 7fc7fbfff640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc80006be60 0x7fc80006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:31.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.508+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fc8040ab4b0 con 0x7fc81406c4d0 2024-04-25T01:01:31.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.508+0000 7fc812ffd640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc80006be60 0x7fc80006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:31.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.509+0000 7fc812ffd640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc80006be60 0x7fc80006e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc8140c2f80 tx=0x7fc808009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:31.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.513+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8140c5050 con 0x7fc81406c4d0 2024-04-25T01:01:31.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:31.687+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc81406d070 con 0x7fc81406c4d0 2024-04-25T01:01:32.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.377+0000 7fc7fbfff640 1 -- 172.21.15.129:0/2370636552 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fc804079a80 con 0x7fc81406c4d0 2024-04-25T01:01:32.379 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' created 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.379+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc80006be60 msgr2=0x7fc80006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.379+0000 7fc819b90640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc80006be60 0x7fc80006e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc8140c2f80 tx=0x7fc808009240 comp rx=0 tx=0).stop 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.379+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 msgr2=0x7fc814072970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.379+0000 7fc819b90640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 0x7fc814072970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc804035040 tx=0x7fc8040046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 shutdown_connections 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc80006be60 0x7fc80006e320 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc814155760 0x7fc814071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc81414f730 0x7fc814071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 --2- 172.21.15.129:0/2370636552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc81406c4d0 0x7fc814072970 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 >> 172.21.15.129:0/2370636552 conn(0x7fc8140a86b0 msgr2=0x7fc8140c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.380+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 shutdown_connections 2024-04-25T01:01:32.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.381+0000 7fc819b90640 1 -- 172.21.15.129:0/2370636552 wait complete. 2024-04-25T01:01:32.405 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T01:01:32.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.502+0000 7f1e57c22640 1 Processor -- start 2024-04-25T01:01:32.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e57c22640 1 -- start start 2024-04-25T01:01:32.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e57c22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e50155d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e57c22640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e500733a0 con 0x7f1e50067300 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e57c22640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e50073680 con 0x7f1e5014f260 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e57c22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1e50158c50 con 0x7f1e500c0620 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e551bd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f1e50067300 0x7f1e500c1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:51940/0 (socket says 172.21.15.129:51940) 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.503+0000 7f1e551bd640 1 -- 172.21.15.129:0/2177952308 learned_addr learned my addr 172.21.15.129:0/2177952308 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e50155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2739209132 0 0) 0x7f1e500733a0 con 0x7f1e50067300 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e3c003520 con 0x7f1e50067300 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5454646 0 0) 0x7f1e3c003520 con 0x7f1e50067300 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 msgr2=0x7f1e50155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e50155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 >> v1:172.21.15.171:6789/0 conn(0x7f1e5014f260 legacy=0x7f1e501590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501588e0 con 0x7f1e50067300 2024-04-25T01:01:32.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f1e40001670 con 0x7f1e50067300 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 >> v1:172.21.15.129:6789/0 conn(0x7f1e50067300 legacy=0x7f1e500c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e3c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e3c003f60 0x7f1e3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e3c004d40 0x7f1e3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50158c50 con 0x7f1e3c004d40 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c00bb50 con 0x7f1e3c003f60 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3c00bd50 con 0x7f1e500c0620 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e3c004d40 0x7f1e3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e551bd640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e3c003f60 0x7f1e3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e559be640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e3c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 -- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 msgr2=0x7f1e3c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e3c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 -- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e3c003f60 msgr2=0x7f1e3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e3c003f60 0x7f1e3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e561bf640 1 -- 172.21.15.129:0/2177952308 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c003520 con 0x7f1e3c004d40 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.504+0000 7f1e559be640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e500c0620 0x7f1e3c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.505+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e3c004d40 0x7f1e3c0051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ac0 tx=0x7f1e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a62ba86a608220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.505+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48036070 con 0x7f1e3c004d40 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.505+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e48031400 con 0x7f1e3c004d40 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.505+0000 7f1e549bc640 1 -- 172.21.15.129:0/2177952308 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48009690 con 0x7f1e3c004d40 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e57c22640 1 -- 172.21.15.129:0/2177952308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e3c004d40 msgr2=0x7f1e3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2177952308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e3c004d40 0x7f1e3c0051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ac0 tx=0x7f1e3c005830 comp rx=0 tx=0).stop 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e561bf640 1 -- 172.21.15.129:0/2177952308 reap_dead start 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e57c22640 1 -- 172.21.15.129:0/2177952308 shutdown_connections 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e57c22640 1 -- 172.21.15.129:0/2177952308 >> 172.21.15.129:0/2177952308 conn(0x7f1e500c71f0 msgr2=0x7f1e500c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e57c22640 1 -- 172.21.15.129:0/2177952308 shutdown_connections 2024-04-25T01:01:32.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.506+0000 7f1e57c22640 1 -- 172.21.15.129:0/2177952308 wait complete. 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 Processor -- start 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 -- start start 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e3c004d40 0x7f1e5014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e500abd10 0x7f1e501eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 0x7f1e501eb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e500c2300 con 0x7f1e501eede0 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5006b400 con 0x7f1e500abd10 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e57c22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1e500c2480 con 0x7f1e3c004d40 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.507+0000 7f1e561bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 0x7f1e501eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 0x7f1e501eb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39478/0 (socket says 172.21.15.129:39478) 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 -- 172.21.15.129:0/2362441830 learned_addr learned my addr 172.21.15.129:0/2362441830 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:32.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 -- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e3c004d40 msgr2=0x7f1e5014e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e3c004d40 0x7f1e5014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 -- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e500abd10 msgr2=0x7f1e501eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e500abd10 0x7f1e501eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 -- 172.21.15.129:0/2362441830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e500ac210 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e561bf640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 0x7f1e501eb130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009be0 tx=0x7f1e48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48036040 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e48043070 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501ee300 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48009690 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.508+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501ee8c0 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.509+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e50075550 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.510+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e48039070 con 0x7f1e501eede0 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.510+0000 7f1e467fc640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1e3406be30 0x7f1e3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:32.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.510+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f1e480aaa00 con 0x7f1e501eede0 2024-04-25T01:01:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.510+0000 7f1e559be640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1e3406be30 0x7f1e3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:32.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.511+0000 7f1e559be640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1e3406be30 0x7f1e3406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e5014a220 tx=0x7f1e4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:32.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.514+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e4804c020 con 0x7f1e501eede0 2024-04-25T01:01:32.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:32.728+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1e501eea90 con 0x7f1e501eede0 2024-04-25T01:01:33.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.384+0000 7f1e467fc640 1 -- 172.21.15.129:0/2362441830 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1e480794a0 con 0x7f1e501eede0 2024-04-25T01:01:33.387 INFO:teuthology.orchestra.run.smithi129.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.394+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1e3406be30 msgr2=0x7f1e3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.394+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1e3406be30 0x7f1e3406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e5014a220 tx=0x7f1e4c009040 comp rx=0 tx=0).stop 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.394+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 msgr2=0x7f1e501eb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.394+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 0x7f1e501eb130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009be0 tx=0x7f1e48004730 comp rx=0 tx=0).stop 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.395+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 shutdown_connections 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.395+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1e3406be30 0x7f1e3406e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.395+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e501eede0 0x7f1e501eb130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.395+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e500abd10 0x7f1e501eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.395+0000 7f1e57c22640 1 --2- 172.21.15.129:0/2362441830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1e3c004d40 0x7f1e5014e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.395+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 >> 172.21.15.129:0/2362441830 conn(0x7f1e500c71f0 msgr2=0x7f1e501498a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:33.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.397+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 shutdown_connections 2024-04-25T01:01:33.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.399+0000 7f1e57c22640 1 -- 172.21.15.129:0/2362441830 wait complete. 2024-04-25T01:01:33.419 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T01:01:33.419 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T01:01:33.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 Processor -- start 2024-04-25T01:01:33.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 -- start start 2024-04-25T01:01:33.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f00b40abc50 0x7f00b40ac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f00b40c37b0 0x7f00b41567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f00b40c2de0 con 0x7f00b40abc50 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f00b4153f90 con 0x7f00b40c37b0 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00ba7fb640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f00b40c24b0 con 0x7f00b406e0a0 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f00b40abc50 0x7f00b40ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f00b40abc50 0x7f00b40ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39494/0 (socket says 172.21.15.129:39494) 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 -- 172.21.15.129:0/4034188215 learned_addr learned my addr 172.21.15.129:0/4034188215 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b37fe640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b8d98640 1 --2- 172.21.15.129:0/4034188215 >> v2:172.21.15.171:3300/0 conn(0x7f00b40c37b0 0x7f00b41567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 -- 172.21.15.129:0/4034188215 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 msgr2=0x7f00b40c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 -- 172.21.15.129:0/4034188215 >> v2:172.21.15.171:3300/0 conn(0x7f00b40c37b0 msgr2=0x7f00b41567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 --2- 172.21.15.129:0/4034188215 >> v2:172.21.15.171:3300/0 conn(0x7f00b40c37b0 0x7f00b41567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b3fff640 1 -- 172.21.15.129:0/4034188215 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b40c2790 con 0x7f00b40abc50 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b37fe640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.524+0000 7f00b8d98640 1 --2- 172.21.15.129:0/4034188215 >> v2:172.21.15.171:3300/0 conn(0x7f00b40c37b0 0x7f00b41567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00b3fff640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40abc50 0x7f00b40ac030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b406c390 tx=0x7f00a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dfef48c22dda17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:33.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00b2ffd640 1 -- 172.21.15.129:0/4034188215 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4037070 con 0x7f00b40abc50 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00b2ffd640 1 -- 172.21.15.129:0/4034188215 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4032440 con 0x7f00b40abc50 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00b2ffd640 1 -- 172.21.15.129:0/4034188215 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4032810 con 0x7f00b40abc50 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 -- 172.21.15.129:0/4034188215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40abc50 msgr2=0x7f00b40ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40abc50 0x7f00b40ac030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b406c390 tx=0x7f00a402a140 comp rx=0 tx=0).stop 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 -- 172.21.15.129:0/4034188215 shutdown_connections 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/4034188215 >> v2:172.21.15.171:3300/0 conn(0x7f00b40c37b0 0x7f00b41567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/4034188215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40abc50 0x7f00b40ac030 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.525+0000 7f00ba7fb640 1 -- 172.21.15.129:0/4034188215 >> 172.21.15.129:0/4034188215 conn(0x7f00b40c6890 msgr2=0x7f00b40c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:33.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.526+0000 7f00ba7fb640 1 -- 172.21.15.129:0/4034188215 shutdown_connections 2024-04-25T01:01:33.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.526+0000 7f00ba7fb640 1 -- 172.21.15.129:0/4034188215 wait complete. 2024-04-25T01:01:33.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 Processor -- start 2024-04-25T01:01:33.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 -- start start 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00b40abc50 0x7f00b4156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 0x7f00b41ec830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00b40c63e0 con 0x7f00b40c37b0 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4153c10 con 0x7f00b40abc50 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00ba7fb640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f00b40c6560 con 0x7f00b406e0a0 2024-04-25T01:01:33.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00b8d98640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 0x7f00b41ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00b8d98640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 0x7f00b41ec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39496/0 (socket says 172.21.15.129:39496) 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.527+0000 7f00b8d98640 1 -- 172.21.15.129:0/1505909430 learned_addr learned my addr 172.21.15.129:0/1505909430 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b3fff640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b37fe640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00b40abc50 0x7f00b4156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b8d98640 1 -- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 msgr2=0x7f00b40a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b8d98640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b8d98640 1 -- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00b40abc50 msgr2=0x7f00b4156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:33.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b8d98640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00b40abc50 0x7f00b4156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b8d98640 1 -- 172.21.15.129:0/1505909430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b414cbc0 con 0x7f00b40c37b0 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b37fe640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00b40abc50 0x7f00b4156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b8d98640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 0x7f00b41ec830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f00ac007df0 tx=0x7f00ac00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac01a070 con 0x7f00b40c37b0 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac003990 con 0x7f00b40c37b0 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b41ed2b0 con 0x7f00b40c37b0 2024-04-25T01:01:33.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac003d80 con 0x7f00b40c37b0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.528+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b414d410 con 0x7f00b40c37b0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.529+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b4068fd0 con 0x7f00b40c37b0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.529+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00ac01d070 con 0x7f00b40c37b0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.530+0000 7f00b0ff9640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a006be30 0x7f00a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.530+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00ac08b050 con 0x7f00b40c37b0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.530+0000 7f00b3fff640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a006be30 0x7f00a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.530+0000 7f00b3fff640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a006be30 0x7f00a006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00a4034000 tx=0x7f00a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:33.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.534+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00ac010070 con 0x7f00b40c37b0 2024-04-25T01:01:33.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:33.716+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f00b406d720 con 0x7f00b40c37b0 2024-04-25T01:01:34.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.385+0000 7f00b0ff9640 1 -- 172.21.15.129:0/1505909430 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f00ac059a90 con 0x7f00b40c37b0 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a006be30 msgr2=0x7f00a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a006be30 0x7f00a006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00a4034000 tx=0x7f00a402e000 comp rx=0 tx=0).stop 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 msgr2=0x7f00b41ec830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 0x7f00b41ec830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f00ac007df0 tx=0x7f00ac00d4a0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 shutdown_connections 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a006be30 0x7f00a006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00b40c37b0 0x7f00b41ec830 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00b40abc50 0x7f00b4156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 --2- 172.21.15.129:0/1505909430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00b406e0a0 0x7f00b40a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 >> 172.21.15.129:0/1505909430 conn(0x7f00b40c6890 msgr2=0x7f00b4071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:34.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 shutdown_connections 2024-04-25T01:01:34.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.389+0000 7f00ba7fb640 1 -- 172.21.15.129:0/1505909430 wait complete. 2024-04-25T01:01:34.404 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:01:34.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.510+0000 7f95df09a640 1 Processor -- start 2024-04-25T01:01:34.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.510+0000 7f95df09a640 1 -- start start 2024-04-25T01:01:34.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95df09a640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f95d80c46e0 0x7f95d8072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:34.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95df09a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f95d80c3790 0x7f95d8072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95df09a640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d8156e20 con 0x7f95d8156940 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95df09a640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f95d8157100 con 0x7f95d80c46e0 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95df09a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f95d81573e0 con 0x7f95d80c3790 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95dce36640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f95d80c46e0 0x7f95d8072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95dce36640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f95d80c46e0 0x7f95d8072650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39502/0 (socket says 172.21.15.129:39502) 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95dce36640 1 -- 172.21.15.129:0/682718708 learned_addr learned my addr 172.21.15.129:0/682718708 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.511+0000 7f95d7fff640 1 --2- 172.21.15.129:0/682718708 >> v2:172.21.15.171:3300/0 conn(0x7f95d80c3790 0x7f95d8072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95dce36640 1 -- 172.21.15.129:0/682718708 >> v2:172.21.15.171:3300/0 conn(0x7f95d80c3790 msgr2=0x7f95d8072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95dce36640 1 --2- 172.21.15.129:0/682718708 >> v2:172.21.15.171:3300/0 conn(0x7f95d80c3790 0x7f95d8072b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95dce36640 1 -- 172.21.15.129:0/682718708 >> v1:172.21.15.171:6789/0 conn(0x7f95d8156940 legacy=0x7f95d806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95dce36640 1 -- 172.21.15.129:0/682718708 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81576c0 con 0x7f95d80c46e0 2024-04-25T01:01:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95dce36640 1 --2- 172.21.15.129:0/682718708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d8072650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d80a7130 tx=0x7f95c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0f2437a7ea311b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95d77fe640 1 -- 172.21.15.129:0/682718708 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8037070 con 0x7f95d80c46e0 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95d77fe640 1 -- 172.21.15.129:0/682718708 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c8032440 con 0x7f95d80c46e0 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.512+0000 7f95d77fe640 1 -- 172.21.15.129:0/682718708 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8032810 con 0x7f95d80c46e0 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 -- 172.21.15.129:0/682718708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 msgr2=0x7f95d8072650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 --2- 172.21.15.129:0/682718708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d8072650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95d80a7130 tx=0x7f95c802a1a0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 -- 172.21.15.129:0/682718708 shutdown_connections 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 --2- 172.21.15.129:0/682718708 >> v2:172.21.15.171:3300/0 conn(0x7f95d80c3790 0x7f95d8072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 --2- 172.21.15.129:0/682718708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d8072650 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 -- 172.21.15.129:0/682718708 >> 172.21.15.129:0/682718708 conn(0x7f95d80c7a80 msgr2=0x7f95d80c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 -- 172.21.15.129:0/682718708 shutdown_connections 2024-04-25T01:01:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.513+0000 7f95df09a640 1 -- 172.21.15.129:0/682718708 wait complete. 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.514+0000 7f95df09a640 1 Processor -- start 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.514+0000 7f95df09a640 1 -- start start 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95d80c3790 0x7f95d80aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d80a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95d8156940 0x7f95d81eb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8073250 con 0x7f95d80c46e0 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f95d80730d0 con 0x7f95d8156940 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f95d80c4ac0 con 0x7f95d80c3790 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d80a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:34.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d80a80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39508/0 (socket says 172.21.15.129:39508) 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 -- 172.21.15.129:0/1607267607 learned_addr learned my addr 172.21.15.129:0/1607267607 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95dd637640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95d8156940 0x7f95d81eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95dce36640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95d80c3790 0x7f95d80aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 -- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95d80c3790 msgr2=0x7f95d80aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95d80c3790 0x7f95d80aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 -- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95d8156940 msgr2=0x7f95d81eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95d8156940 0x7f95d81eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 -- 172.21.15.129:0/1607267607 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d8153a00 con 0x7f95d80c46e0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d7fff640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d80a80d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00d800 tx=0x7f95cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004030 con 0x7f95d80c46e0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc0041d0 con 0x7f95d80c46e0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d8153440 con 0x7f95d80c46e0 2024-04-25T01:01:34.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00b6c0 con 0x7f95d80c46e0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.515+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81eb060 con 0x7f95d80c46e0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.516+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d80681a0 con 0x7f95d80c46e0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.517+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95cc00b860 con 0x7f95d80c46e0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.517+0000 7f95d57fa640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f95c406be60 0x7f95c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.517+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f95cc08b0b0 con 0x7f95d80c46e0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.517+0000 7f95dce36640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f95c406be60 0x7f95c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.518+0000 7f95dce36640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f95c406be60 0x7f95c406e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95c8034000 tx=0x7f95c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:34.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.521+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f95cc0045c0 con 0x7f95d80c46e0 2024-04-25T01:01:34.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:34.698+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f95d80a7c00 con 0x7f95d80c46e0 2024-04-25T01:01:36.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.406+0000 7f95d57fa640 1 -- 172.21.15.129:0/1607267607 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f95cc059370 con 0x7f95d80c46e0 2024-04-25T01:01:36.406 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data_ec' created 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f95c406be60 msgr2=0x7f95c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f95c406be60 0x7f95c406e320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95c8034000 tx=0x7f95c802e000 comp rx=0 tx=0).stop 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 msgr2=0x7f95d80a80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d80a80d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00d800 tx=0x7f95cc00dcc0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 shutdown_connections 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f95c406be60 0x7f95c406e320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95d8156940 0x7f95d81eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f95d80c46e0 0x7f95d80a80d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 --2- 172.21.15.129:0/1607267607 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95d80c3790 0x7f95d80aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 >> 172.21.15.129:0/1607267607 conn(0x7f95d80c7a80 msgr2=0x7f95d80a74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:36.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 shutdown_connections 2024-04-25T01:01:36.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.409+0000 7f95df09a640 1 -- 172.21.15.129:0/1607267607 wait complete. 2024-04-25T01:01:36.428 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T01:01:36.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.533+0000 7ff210493640 1 Processor -- start 2024-04-25T01:01:36.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.533+0000 7ff210493640 1 -- start start 2024-04-25T01:01:36.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff210493640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 0x7ff2080c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff210493640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff20806cfe0 con 0x7ff2080c8b20 2024-04-25T01:01:36.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff210493640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff208155490 con 0x7ff2080c9030 2024-04-25T01:01:36.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff210493640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff208155dc0 con 0x7ff208152bb0 2024-04-25T01:01:36.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff20ea30640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff2080c8b20 0x7ff20815fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:51946/0 (socket says 172.21.15.129:51946) 2024-04-25T01:01:36.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff20ea30640 1 -- 172.21.15.129:0/2344622757 learned_addr learned my addr 172.21.15.129:0/2344622757 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff20e22f640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 0x7ff2080c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2635819473 0 0) 0x7ff20806cfe0 con 0x7ff2080c8b20 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.534+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1f4003520 con 0x7ff2080c8b20 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1489919930 0 0) 0x7ff1f4003520 con 0x7ff2080c8b20 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 msgr2=0x7ff2080c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 0x7ff2080c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 >> v1:172.21.15.171:6789/0 conn(0x7ff2080c9030 legacy=0x7ff208158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff208155770 con 0x7ff2080c8b20 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7ff2000019f0 con 0x7ff2080c8b20 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 >> v1:172.21.15.129:6789/0 conn(0x7ff2080c8b20 legacy=0x7ff20815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 0x7ff1f40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1f4003f60 0x7ff1f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f4004d40 0x7ff1f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff208155dc0 con 0x7ff1f4003f60 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f400bb50 con 0x7ff1f4004d40 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f400bd50 con 0x7ff208152bb0 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20da2e640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1f4003f60 0x7ff1f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20e22f640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 0x7ff1f40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20ea30640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f4004d40 0x7ff1f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20da2e640 1 -- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 msgr2=0x7ff1f40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20da2e640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff208152bb0 0x7ff1f40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20da2e640 1 -- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f4004d40 msgr2=0x7ff1f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20da2e640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f4004d40 0x7ff1f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.535+0000 7ff20da2e640 1 -- 172.21.15.129:0/2344622757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f4003520 con 0x7ff1f4003f60 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.536+0000 7ff20da2e640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1f4003f60 0x7ff1f4004800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8001750 tx=0x7ff1f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a860a1001eab75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.536+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8001370 con 0x7ff1f4003f60 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.536+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f80052d0 con 0x7ff1f4003f60 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.536+0000 7ff20d22d640 1 -- 172.21.15.129:0/2344622757 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f802abe0 con 0x7ff1f4003f60 2024-04-25T01:01:36.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff210493640 1 -- 172.21.15.129:0/2344622757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1f4003f60 msgr2=0x7ff1f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff210493640 1 --2- 172.21.15.129:0/2344622757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1f4003f60 0x7ff1f4004800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8001750 tx=0x7ff1f4005830 comp rx=0 tx=0).stop 2024-04-25T01:01:36.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff20ea30640 1 -- 172.21.15.129:0/2344622757 reap_dead start 2024-04-25T01:01:36.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff210493640 1 -- 172.21.15.129:0/2344622757 shutdown_connections 2024-04-25T01:01:36.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff210493640 1 -- 172.21.15.129:0/2344622757 >> 172.21.15.129:0/2344622757 conn(0x7ff2080acbf0 msgr2=0x7ff2080ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:36.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff210493640 1 -- 172.21.15.129:0/2344622757 shutdown_connections 2024-04-25T01:01:36.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.537+0000 7ff210493640 1 -- 172.21.15.129:0/2344622757 wait complete. 2024-04-25T01:01:36.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 Processor -- start 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 -- start start 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1f4003f60 0x7ff2080b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2081485c0 0x7ff208151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 0x7ff2081ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2080c94f0 con 0x7ff2081eecb0 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff208156250 con 0x7ff2081485c0 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff210493640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff20806d3d0 con 0x7ff1f4003f60 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.538+0000 7ff20ea30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 0x7ff2081ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 0x7ff2081ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39534/0 (socket says 172.21.15.129:39534) 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 -- 172.21.15.129:0/467252620 learned_addr learned my addr 172.21.15.129:0/467252620 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 -- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1f4003f60 msgr2=0x7ff2080b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1f4003f60 0x7ff2080b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 -- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2081485c0 msgr2=0x7ff208151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:36.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2081485c0 0x7ff208151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 -- 172.21.15.129:0/467252620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2081597b0 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff20ea30640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 0x7ff2081ef130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2080caae0 tx=0x7ff200007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000ab60 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff200019070 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2081eb360 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000133f0 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.539+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2080ae370 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.540+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff208068fd0 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.540+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff200013590 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.541+0000 7ff1feffd640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.541+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7ff20008b760 con 0x7ff2081eecb0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.541+0000 7ff20e22f640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:36.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.542+0000 7ff20e22f640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff204009ab0 tx=0x7ff2040095d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:36.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.546+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff200059df0 con 0x7ff2081eecb0 2024-04-25T01:01:36.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:36.717+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff2080b0ba0 con 0x7ff2081eecb0 2024-04-25T01:01:37.236 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:01:37.235+0000 7f2242407640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:01:37.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.410+0000 7ff1feffd640 1 -- 172.21.15.129:0/467252620 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7ff2000596f0 con 0x7ff2081eecb0 2024-04-25T01:01:37.411 INFO:teuthology.orchestra.run.smithi129.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff1ec06be30 msgr2=0x7ff1ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff204009ab0 tx=0x7ff2040095d0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 msgr2=0x7ff2081ef130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 0x7ff2081ef130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2080caae0 tx=0x7ff200007b20 comp rx=0 tx=0).stop 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 shutdown_connections 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2081eecb0 0x7ff2081ef130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2081485c0 0x7ff208151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 --2- 172.21.15.129:0/467252620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1f4003f60 0x7ff2080b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 >> 172.21.15.129:0/467252620 conn(0x7ff2080acbf0 msgr2=0x7ff2080c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:37.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 shutdown_connections 2024-04-25T01:01:37.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.412+0000 7ff210493640 1 -- 172.21.15.129:0/467252620 wait complete. 2024-04-25T01:01:37.437 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T01:01:37.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.538+0000 7f42055dc640 1 Processor -- start 2024-04-25T01:01:37.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.538+0000 7f42055dc640 1 -- start start 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f42055dc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f42000c7930 0x7f42000a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f42055dc640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f4200155e90 0x7f42000c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f42055dc640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42000c7710 con 0x7f42000c6440 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f42055dc640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f42000c69b0 con 0x7f42000c7930 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f42055dc640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f42000c7280 con 0x7f4200155e90 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41feffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f42000c7930 0x7f42000a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41fe7fc640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f42000c6440 0x7f4200155780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60136/0 (socket says 172.21.15.129:60136) 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41feffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f42000c7930 0x7f42000a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34312/0 (socket says 172.21.15.129:34312) 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41fe7fc640 1 -- 172.21.15.129:0/617556112 learned_addr learned my addr 172.21.15.129:0/617556112 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41ff7fe640 1 --2- 172.21.15.129:0/617556112 >> v2:172.21.15.171:3300/0 conn(0x7f4200155e90 0x7f42000c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:37.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41feffd640 1 -- 172.21.15.129:0/617556112 >> v2:172.21.15.171:3300/0 conn(0x7f4200155e90 msgr2=0x7f42000c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41feffd640 1 --2- 172.21.15.129:0/617556112 >> v2:172.21.15.171:3300/0 conn(0x7f4200155e90 0x7f42000c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41feffd640 1 -- 172.21.15.129:0/617556112 >> v1:172.21.15.129:6789/0 conn(0x7f42000c6440 legacy=0x7f4200155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41feffd640 1 -- 172.21.15.129:0/617556112 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42000c2f30 con 0x7f42000c7930 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41fdffb640 1 -- 172.21.15.129:0/617556112 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120899220 0 0) 0x7f42000c7710 con 0x7f42000c6440 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.539+0000 7f41ff7fe640 1 --2- 172.21.15.129:0/617556112 >> v2:172.21.15.171:3300/0 conn(0x7f4200155e90 0x7f42000c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.540+0000 7f41feffd640 1 --2- 172.21.15.129:0/617556112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c7930 0x7f42000a6ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42000c89e0 tx=0x7f41f002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4450c5c1182e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.540+0000 7f41fdffb640 1 -- 172.21.15.129:0/617556112 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007740 con 0x7f42000c7930 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.540+0000 7f41fdffb640 1 -- 172.21.15.129:0/617556112 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f00078e0 con 0x7f42000c7930 2024-04-25T01:01:37.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.540+0000 7f41fdffb640 1 -- 172.21.15.129:0/617556112 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007cb0 con 0x7f42000c7930 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 -- 172.21.15.129:0/617556112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c7930 msgr2=0x7f42000a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 --2- 172.21.15.129:0/617556112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c7930 0x7f42000a6ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f42000c89e0 tx=0x7f41f002a560 comp rx=0 tx=0).stop 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 -- 172.21.15.129:0/617556112 shutdown_connections 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 --2- 172.21.15.129:0/617556112 >> v2:172.21.15.171:3300/0 conn(0x7f4200155e90 0x7f42000c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 --2- 172.21.15.129:0/617556112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c7930 0x7f42000a6ce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 -- 172.21.15.129:0/617556112 >> 172.21.15.129:0/617556112 conn(0x7f42000a9730 msgr2=0x7f42000a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 -- 172.21.15.129:0/617556112 shutdown_connections 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.541+0000 7f42055dc640 1 -- 172.21.15.129:0/617556112 wait complete. 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.542+0000 7f42055dc640 1 Processor -- start 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.542+0000 7f42055dc640 1 -- start start 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 0x7f4200151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:37.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42000c7930 0x7f420014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4200155e90 0x7f420014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42000a7330 con 0x7f42000c6440 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42000c2c50 con 0x7f42000c7930 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f42000a74b0 con 0x7f4200155e90 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 0x7f4200151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 0x7f4200151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34318/0 (socket says 172.21.15.129:34318) 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 -- 172.21.15.129:0/3108783440 learned_addr learned my addr 172.21.15.129:0/3108783440 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41fe7fc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42000c7930 0x7f420014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41ff7fe640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4200155e90 0x7f420014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 -- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4200155e90 msgr2=0x7f420014b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4200155e90 0x7f420014b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 -- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42000c7930 msgr2=0x7f420014aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42000c7930 0x7f420014aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 -- 172.21.15.129:0/3108783440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420014a0e0 con 0x7f42000c6440 2024-04-25T01:01:37.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41feffd640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 0x7f4200151600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41f002ab50 tx=0x7f41f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:37.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0040070 con 0x7f42000c6440 2024-04-25T01:01:37.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f0031c40 con 0x7f42000c6440 2024-04-25T01:01:37.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f003c420 con 0x7f42000c6440 2024-04-25T01:01:37.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.543+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4200150440 con 0x7f42000c6440 2024-04-25T01:01:37.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.544+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4200150a00 con 0x7f42000c6440 2024-04-25T01:01:37.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.544+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200075550 con 0x7f42000c6440 2024-04-25T01:01:37.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.545+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41f0039040 con 0x7f42000c6440 2024-04-25T01:01:37.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.545+0000 7f41e77fe640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f41ec06be30 0x7f41ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:37.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.546+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f41f00ac6e0 con 0x7f42000c6440 2024-04-25T01:01:37.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.546+0000 7f41fe7fc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f41ec06be30 0x7f41ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:37.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.546+0000 7f41fe7fc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f41ec06be30 0x7f41ec06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42000c1f50 tx=0x7f41f4007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:37.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.550+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f41f007a720 con 0x7f42000c6440 2024-04-25T01:01:37.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:37.762+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f42000c7710 con 0x7f42000c6440 2024-04-25T01:01:38.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.411+0000 7f41e77fe640 1 -- 172.21.15.129:0/3108783440 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f41f007ea20 con 0x7f42000c6440 2024-04-25T01:01:38.412 INFO:teuthology.orchestra.run.smithi129.stderr:added data pool 4 to fsmap 2024-04-25T01:01:38.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f41ec06be30 msgr2=0x7f41ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f41ec06be30 0x7f41ec06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42000c1f50 tx=0x7f41f4007d50 comp rx=0 tx=0).stop 2024-04-25T01:01:38.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 msgr2=0x7f4200151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 0x7f4200151600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41f002ab50 tx=0x7f41f0004a40 comp rx=0 tx=0).stop 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 shutdown_connections 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f41ec06be30 0x7f41ec06e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4200155e90 0x7f420014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42000c7930 0x7f420014aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 --2- 172.21.15.129:0/3108783440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42000c6440 0x7f4200151600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 >> 172.21.15.129:0/3108783440 conn(0x7f42000a9730 msgr2=0x7f420006f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 shutdown_connections 2024-04-25T01:01:38.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.415+0000 7f42055dc640 1 -- 172.21.15.129:0/3108783440 wait complete. 2024-04-25T01:01:38.429 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:01:38.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.531+0000 7f78ec1b7640 1 Processor -- start 2024-04-25T01:01:38.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.531+0000 7f78ec1b7640 1 -- start start 2024-04-25T01:01:38.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.531+0000 7f78ec1b7640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f78e40c6110 0x7f78e4153f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.531+0000 7f78ec1b7640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78e40c8070 con 0x7f78e40a6ec0 2024-04-25T01:01:38.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.531+0000 7f78ec1b7640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78e40c59d0 con 0x7f78e406ef80 2024-04-25T01:01:38.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.531+0000 7f78ec1b7640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f78e4156930 con 0x7f78e40c6110 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e9752640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f78e40a6ec0 0x7f78e40c3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60146/0 (socket says 172.21.15.129:60146) 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e9752640 1 -- 172.21.15.129:0/2261504567 learned_addr learned my addr 172.21.15.129:0/2261504567 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124828523 0 0) 0x7f78e40c8070 con 0x7f78e40a6ec0 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78d0003520 con 0x7f78e40a6ec0 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78ea754640 1 --2- 172.21.15.129:0/2261504567 >> v2:172.21.15.171:3300/0 conn(0x7f78e40c6110 0x7f78e4153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2875600152 0 0) 0x7f78d0003520 con 0x7f78e40a6ec0 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 >> v2:172.21.15.171:3300/0 conn(0x7f78e40c6110 msgr2=0x7f78e4153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 --2- 172.21.15.129:0/2261504567 >> v2:172.21.15.171:3300/0 conn(0x7f78e40c6110 0x7f78e4153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 >> v1:172.21.15.171:6789/0 conn(0x7f78e406ef80 legacy=0x7f78e4153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e40c5cb0 con 0x7f78e40a6ec0 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f78d4001670 con 0x7f78e40a6ec0 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 >> v1:172.21.15.129:6789/0 conn(0x7f78e40a6ec0 legacy=0x7f78e40c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78d00052e0 0x7f78d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78d000c180 con 0x7f78d0004800 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f78d000c330 con 0x7f78d00039b0 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f78d000c530 con 0x7f78d00052e0 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e9752640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78e9f53640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.532+0000 7f78ea754640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78d00052e0 0x7f78d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e9752640 1 -- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78d00052e0 msgr2=0x7f78d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e9752640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78d00052e0 0x7f78d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e9752640 1 -- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78d00039b0 msgr2=0x7f78d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e9752640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e9752640 1 -- 172.21.15.129:0/2261504567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d0003520 con 0x7f78d0004800 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78ea754640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78d00052e0 0x7f78d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78ea754640 1 -- 172.21.15.129:0/2261504567 reap_dead start 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e9752640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78d0004c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78d400c210 tx=0x7f78d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857589c7b97d3e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:38.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4007d10 con 0x7f78d0004800 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d4005ad0 con 0x7f78d0004800 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.533+0000 7f78e8f51640 1 -- 172.21.15.129:0/2261504567 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4007070 con 0x7f78d0004800 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2261504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 msgr2=0x7f78d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78d0004c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78d400c210 tx=0x7f78d0005830 comp rx=0 tx=0).stop 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2261504567 shutdown_connections 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2261504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78d0004c80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2261504567 >> 172.21.15.129:0/2261504567 conn(0x7f78e40a9730 msgr2=0x7f78e40a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2261504567 shutdown_connections 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.534+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2261504567 wait complete. 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.535+0000 7f78ec1b7640 1 Processor -- start 2024-04-25T01:01:38.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.535+0000 7f78ec1b7640 1 -- start start 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78e41514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78e41dc9a0 0x7f78e41deab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e41ec5d0 0x7f78e41deff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4150c90 con 0x7f78d0004800 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f78e40c8350 con 0x7f78e41ec5d0 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f78e4150e10 con 0x7f78e41dc9a0 2024-04-25T01:01:38.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78e41514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78e41514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34338/0 (socket says 172.21.15.129:34338) 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 -- 172.21.15.129:0/2851285752 learned_addr learned my addr 172.21.15.129:0/2851285752 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9752640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78e41dc9a0 0x7f78e41deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ea754640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e41ec5d0 0x7f78e41deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 -- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78e41dc9a0 msgr2=0x7f78e41deab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78e41dc9a0 0x7f78e41deab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 -- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e41ec5d0 msgr2=0x7f78e41deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e41ec5d0 0x7f78e41deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 -- 172.21.15.129:0/2851285752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e41dcef0 con 0x7f78d0004800 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78e9f53640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78e41514a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00f990 tx=0x7f78dc00fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc003b80 con 0x7f78d0004800 2024-04-25T01:01:38.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78dc018070 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e41dd1d0 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0050d0 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.536+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41dd790 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.537+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e4075550 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.538+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78dc005270 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.538+0000 7f78da7fc640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f78c806be30 0x7f78c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.538+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f78dc08b0e0 con 0x7f78d0004800 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.538+0000 7f78e9752640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f78c806be30 0x7f78c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.539+0000 7f78e9752640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f78c806be30 0x7f78c806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78d4032000 tx=0x7f78d4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:38.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.542+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f78dc059100 con 0x7f78d0004800 2024-04-25T01:01:38.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.753+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e4152bd0 con 0x7f78d0004800 2024-04-25T01:01:38.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.754+0000 7f78da7fc640 1 -- 172.21.15.129:0/2851285752 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f78dc05d400 con 0x7f78d0004800 2024-04-25T01:01:38.754 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:38.755 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:37.763659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:01:38.755 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T01:01:38.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f78c806be30 msgr2=0x7f78c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f78c806be30 0x7f78c806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78d4032000 tx=0x7f78d4009030 comp rx=0 tx=0).stop 2024-04-25T01:01:38.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 msgr2=0x7f78e41514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78e41514a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00f990 tx=0x7f78dc00fe50 comp rx=0 tx=0).stop 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 shutdown_connections 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f78c806be30 0x7f78c806e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f78e41dc9a0 0x7f78e41deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e41ec5d0 0x7f78e41deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 --2- 172.21.15.129:0/2851285752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78d0004800 0x7f78e41514a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 >> 172.21.15.129:0/2851285752 conn(0x7f78e40a9730 msgr2=0x7f78e40730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 shutdown_connections 2024-04-25T01:01:38.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.756+0000 7f78ec1b7640 1 -- 172.21.15.129:0/2851285752 wait complete. 2024-04-25T01:01:38.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:38.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.866+0000 7f1626f30640 1 Processor -- start 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.866+0000 7f1626f30640 1 -- start start 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f1626f30640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162014f6f0 0x7f162014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f1626f30640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f16200ac540 0x7f16200c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f1626f30640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16200c5180 con 0x7f162006d6c0 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f1626f30640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f1620155720 con 0x7f16200ac540 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f1626f30640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1620153010 con 0x7f162014f6f0 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f16200ac540 0x7f16200c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f16200ac540 0x7f16200c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34342/0 (socket says 172.21.15.129:34342) 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 -- 172.21.15.129:0/3695486219 learned_addr learned my addr 172.21.15.129:0/3695486219 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:38.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f1624ccc640 1 --2- 172.21.15.129:0/3695486219 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162014f6f0 0x7f162014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 -- 172.21.15.129:0/3695486219 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162014f6f0 msgr2=0x7f162014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 --2- 172.21.15.129:0/3695486219 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162014f6f0 0x7f162014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 -- 172.21.15.129:0/3695486219 >> v1:172.21.15.171:6789/0 conn(0x7f162006d6c0 legacy=0x7f16200c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.867+0000 7f161ffff640 1 -- 172.21.15.129:0/3695486219 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16201559e0 con 0x7f16200ac540 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.868+0000 7f161ffff640 1 --2- 172.21.15.129:0/3695486219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16200ac540 0x7f16200c4800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1614009870 tx=0x7f161402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194189c2d670f6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.868+0000 7f161f7fe640 1 -- 172.21.15.129:0/3695486219 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614037070 con 0x7f16200ac540 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.868+0000 7f161f7fe640 1 -- 172.21.15.129:0/3695486219 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1614032440 con 0x7f16200ac540 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.868+0000 7f161f7fe640 1 -- 172.21.15.129:0/3695486219 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614032810 con 0x7f16200ac540 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 -- 172.21.15.129:0/3695486219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16200ac540 msgr2=0x7f16200c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 --2- 172.21.15.129:0/3695486219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16200ac540 0x7f16200c4800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1614009870 tx=0x7f161402a1c0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 -- 172.21.15.129:0/3695486219 shutdown_connections 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 --2- 172.21.15.129:0/3695486219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16200ac540 0x7f16200c4800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 --2- 172.21.15.129:0/3695486219 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162014f6f0 0x7f162014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 -- 172.21.15.129:0/3695486219 >> 172.21.15.129:0/3695486219 conn(0x7f16200a86b0 msgr2=0x7f16200a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 -- 172.21.15.129:0/3695486219 shutdown_connections 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.869+0000 7f1626f30640 1 -- 172.21.15.129:0/3695486219 wait complete. 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.870+0000 7f1626f30640 1 Processor -- start 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.870+0000 7f1626f30640 1 -- start start 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162006d6c0 0x7f1620154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16200ac540 0x7f16200bf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 0x7f16200bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1620152c30 con 0x7f162014f6f0 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1620155fa0 con 0x7f16200ac540 2024-04-25T01:01:38.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1620152db0 con 0x7f162006d6c0 2024-04-25T01:01:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 0x7f16200bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 0x7f16200bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34356/0 (socket says 172.21.15.129:34356) 2024-04-25T01:01:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 -- 172.21.15.129:0/1949881138 learned_addr learned my addr 172.21.15.129:0/1949881138 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1624ccc640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162006d6c0 0x7f1620154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f161ffff640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16200ac540 0x7f16200bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 -- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162006d6c0 msgr2=0x7f1620154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162006d6c0 0x7f1620154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 -- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16200ac540 msgr2=0x7f16200bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16200ac540 0x7f16200bf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 -- 172.21.15.129:0/1949881138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16201473d0 con 0x7f162014f6f0 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f16254cd640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 0x7f16200bfea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f161800c6b0 tx=0x7f161800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618003b70 con 0x7f162014f6f0 2024-04-25T01:01:38.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1618003d30 con 0x7f162014f6f0 2024-04-25T01:01:38.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1620147670 con 0x7f162014f6f0 2024-04-25T01:01:38.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618005310 con 0x7f162014f6f0 2024-04-25T01:01:38.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.871+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1620147f30 con 0x7f162014f6f0 2024-04-25T01:01:38.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.872+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16200ab510 con 0x7f162014f6f0 2024-04-25T01:01:38.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.873+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16180054b0 con 0x7f162014f6f0 2024-04-25T01:01:38.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.873+0000 7f161d7fa640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f160c06be30 0x7f160c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:38.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.873+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f161808ae20 con 0x7f162014f6f0 2024-04-25T01:01:38.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.874+0000 7f1624ccc640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f160c06be30 0x7f160c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:38.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.874+0000 7f1624ccc640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f160c06be30 0x7f160c06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1610006fd0 tx=0x7f1610008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:38.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:38.878+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1618058ed0 con 0x7f162014f6f0 2024-04-25T01:01:39.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.052+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f162015e870 con 0x7f162014f6f0 2024-04-25T01:01:39.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.053+0000 7f161d7fa640 1 -- 172.21.15.129:0/1949881138 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f161805d1d0 con 0x7f162014f6f0 2024-04-25T01:01:39.054 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:39.054 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f160c06be30 msgr2=0x7f160c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f160c06be30 0x7f160c06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1610006fd0 tx=0x7f1610008040 comp rx=0 tx=0).stop 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 msgr2=0x7f16200bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 0x7f16200bfea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f161800c6b0 tx=0x7f161800cb70 comp rx=0 tx=0).stop 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 shutdown_connections 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f160c06be30 0x7f160c06e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f162014f6f0 0x7f16200bfea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16200ac540 0x7f16200bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 --2- 172.21.15.129:0/1949881138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f162006d6c0 0x7f1620154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 >> 172.21.15.129:0/1949881138 conn(0x7f16200a86b0 msgr2=0x7f162006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 shutdown_connections 2024-04-25T01:01:39.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.055+0000 7f1626f30640 1 -- 172.21.15.129:0/1949881138 wait complete. 2024-04-25T01:01:39.067 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:39.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.163+0000 7f501a238640 1 Processor -- start 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.163+0000 7f501a238640 1 -- start start 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f501a238640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5014070e10 0x7f501406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f501a238640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f50140c6eb0 0x7f50140c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f501a238640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50140c7860 con 0x7f50140c4e50 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f501a238640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f5014153da0 con 0x7f50140c6eb0 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f501a238640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f50140c8350 con 0x7f5014070e10 2024-04-25T01:01:39.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5013fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f50140c6eb0 0x7f50140c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5012ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f50140c4e50 0x7f50140aa6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60162/0 (socket says 172.21.15.129:60162) 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5012ffd640 1 -- 172.21.15.129:0/2836346045 learned_addr learned my addr 172.21.15.129:0/2836346045 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f50137fe640 1 --2- 172.21.15.129:0/2836346045 >> v2:172.21.15.171:3300/0 conn(0x7f5014070e10 0x7f501406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5013fff640 1 -- 172.21.15.129:0/2836346045 >> v2:172.21.15.171:3300/0 conn(0x7f5014070e10 msgr2=0x7f501406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5013fff640 1 --2- 172.21.15.129:0/2836346045 >> v2:172.21.15.171:3300/0 conn(0x7f5014070e10 0x7f501406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5013fff640 1 -- 172.21.15.129:0/2836346045 >> v1:172.21.15.129:6789/0 conn(0x7f50140c4e50 legacy=0x7f50140aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f50127fc640 1 -- 172.21.15.129:0/2836346045 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310670229 0 0) 0x7f50140c7860 con 0x7f50140c4e50 2024-04-25T01:01:39.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f5013fff640 1 -- 172.21.15.129:0/2836346045 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50140c6990 con 0x7f50140c6eb0 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.164+0000 7f50137fe640 1 --2- 172.21.15.129:0/2836346045 >> v2:172.21.15.171:3300/0 conn(0x7f5014070e10 0x7f501406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.165+0000 7f5013fff640 1 --2- 172.21.15.129:0/2836346045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c6eb0 0x7f50140c7320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50140c6040 tx=0x7f500c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea91986fcf1ae179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.165+0000 7f50127fc640 1 -- 172.21.15.129:0/2836346045 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c007740 con 0x7f50140c6eb0 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.165+0000 7f50127fc640 1 -- 172.21.15.129:0/2836346045 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c0078e0 con 0x7f50140c6eb0 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.165+0000 7f50127fc640 1 -- 172.21.15.129:0/2836346045 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c007cb0 con 0x7f50140c6eb0 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 -- 172.21.15.129:0/2836346045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c6eb0 msgr2=0x7f50140c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 --2- 172.21.15.129:0/2836346045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c6eb0 0x7f50140c7320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50140c6040 tx=0x7f500c02a680 comp rx=0 tx=0).stop 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 -- 172.21.15.129:0/2836346045 shutdown_connections 2024-04-25T01:01:39.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 --2- 172.21.15.129:0/2836346045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c6eb0 0x7f50140c7320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 --2- 172.21.15.129:0/2836346045 >> v2:172.21.15.171:3300/0 conn(0x7f5014070e10 0x7f501406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 -- 172.21.15.129:0/2836346045 >> 172.21.15.129:0/2836346045 conn(0x7f50140c5490 msgr2=0x7f50140c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:39.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 -- 172.21.15.129:0/2836346045 shutdown_connections 2024-04-25T01:01:39.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.166+0000 7f501a238640 1 -- 172.21.15.129:0/2836346045 wait complete. 2024-04-25T01:01:39.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.167+0000 7f501a238640 1 Processor -- start 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.167+0000 7f501a238640 1 -- start start 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f501a238640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5014070e10 0x7f5014150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f501a238640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 0x7f50141e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f501a238640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f50140c6eb0 0x7f50141e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f501a238640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50140c8630 con 0x7f50140c4e50 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f501a238640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f50140711f0 con 0x7f5014070e10 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f501a238640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f50140aade0 con 0x7f50140c6eb0 2024-04-25T01:01:39.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 0x7f50141e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 0x7f50141e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34362/0 (socket says 172.21.15.129:34362) 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 -- 172.21.15.129:0/3602667033 learned_addr learned my addr 172.21.15.129:0/3602667033 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f50137fe640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5014070e10 0x7f5014150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5013fff640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f50140c6eb0 0x7f50141e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 -- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f50140c6eb0 msgr2=0x7f50141e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f50140c6eb0 0x7f50141e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 -- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5014070e10 msgr2=0x7f5014150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5014070e10 0x7f5014150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 -- 172.21.15.129:0/3602667033 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50140a97c0 con 0x7f50140c4e50 2024-04-25T01:01:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5013fff640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f50140c6eb0 0x7f50141e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.168+0000 7f5012ffd640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 0x7f50141e8f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c500 tx=0x7f4ffc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.169+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc018070 con 0x7f50140c4e50 2024-04-25T01:01:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.169+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ffc007580 con 0x7f50140c4e50 2024-04-25T01:01:39.178 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.169+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc004a20 con 0x7f50140c4e50 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.169+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50141ecdf0 con 0x7f50140c4e50 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.169+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50141ecb10 con 0x7f50140c4e50 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.170+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5014075550 con 0x7f50140c4e50 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.170+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ffc01b070 con 0x7f50140c4e50 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.170+0000 7f500bfff640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4ff406be60 0x7f4ff406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.171+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4ffc08a360 con 0x7f50140c4e50 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.171+0000 7f50137fe640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4ff406be60 0x7f4ff406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.171+0000 7f50137fe640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4ff406be60 0x7f4ff406e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5004006fd0 tx=0x7f5004008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.175+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ffc058410 con 0x7f50140c4e50 2024-04-25T01:01:39.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.349+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50140c7f50 con 0x7f50140c4e50 2024-04-25T01:01:39.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.351+0000 7f500bfff640 1 -- 172.21.15.129:0/3602667033 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f4ffc05c710 con 0x7f50140c4e50 2024-04-25T01:01:39.352 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:39.352 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4ff406be60 msgr2=0x7f4ff406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4ff406be60 0x7f4ff406e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5004006fd0 tx=0x7f5004008040 comp rx=0 tx=0).stop 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 msgr2=0x7f50141e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 0x7f50141e8f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00c500 tx=0x7f4ffc00c9c0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 shutdown_connections 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4ff406be60 0x7f4ff406e320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f50140c6eb0 0x7f50141e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f50140c4e50 0x7f50141e8f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 --2- 172.21.15.129:0/3602667033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5014070e10 0x7f5014150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 >> 172.21.15.129:0/3602667033 conn(0x7f50140c5490 msgr2=0x7f50141530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 shutdown_connections 2024-04-25T01:01:39.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.353+0000 7f501a238640 1 -- 172.21.15.129:0/3602667033 wait complete. 2024-04-25T01:01:39.381 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:01:39.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.477+0000 7f0e743b0640 1 Processor -- start 2024-04-25T01:01:39.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.477+0000 7f0e743b0640 1 -- start start 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e743b0640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0e6c158da0 0x7f0e6c152810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e743b0640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0e6c152d50 0x7f0e6c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e743b0640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6c159200 con 0x7f0e6c1589c0 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e743b0640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6c0c84e0 con 0x7f0e6c158da0 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e743b0640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6c0c7010 con 0x7f0e6c152d50 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e7214c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0e6c1589c0 0x7f0e6c0c3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60170/0 (socket says 172.21.15.129:60170) 2024-04-25T01:01:39.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e7214c640 1 -- 172.21.15.129:0/531387976 learned_addr learned my addr 172.21.15.129:0/531387976 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e7194b640 1 --2- 172.21.15.129:0/531387976 >> v2:172.21.15.129:3300/0 conn(0x7f0e6c158da0 0x7f0e6c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e7114a640 1 -- 172.21.15.129:0/531387976 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854496633 0 0) 0x7f0e6c159200 con 0x7f0e6c1589c0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.478+0000 7f0e7294d640 1 --2- 172.21.15.129:0/531387976 >> v2:172.21.15.171:3300/0 conn(0x7f0e6c152d50 0x7f0e6c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7114a640 1 -- 172.21.15.129:0/531387976 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e58003520 con 0x7f0e6c1589c0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7194b640 1 -- 172.21.15.129:0/531387976 >> v2:172.21.15.171:3300/0 conn(0x7f0e6c152d50 msgr2=0x7f0e6c0708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7194b640 1 --2- 172.21.15.129:0/531387976 >> v2:172.21.15.171:3300/0 conn(0x7f0e6c152d50 0x7f0e6c0708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7194b640 1 -- 172.21.15.129:0/531387976 >> v1:172.21.15.129:6789/0 conn(0x7f0e6c1589c0 legacy=0x7f0e6c0c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7194b640 1 -- 172.21.15.129:0/531387976 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c06da10 con 0x7f0e6c158da0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7194b640 1 --2- 172.21.15.129:0/531387976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c152810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c072690 tx=0x7f0e5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4170a979114b31b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7114a640 1 -- 172.21.15.129:0/531387976 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c037070 con 0x7f0e6c158da0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7114a640 1 -- 172.21.15.129:0/531387976 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5c032440 con 0x7f0e6c158da0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.479+0000 7f0e7114a640 1 -- 172.21.15.129:0/531387976 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c032810 con 0x7f0e6c158da0 2024-04-25T01:01:39.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 -- 172.21.15.129:0/531387976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 msgr2=0x7f0e6c152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 --2- 172.21.15.129:0/531387976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c152810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c072690 tx=0x7f0e5c02a230 comp rx=0 tx=0).stop 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 -- 172.21.15.129:0/531387976 shutdown_connections 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 --2- 172.21.15.129:0/531387976 >> v2:172.21.15.171:3300/0 conn(0x7f0e6c152d50 0x7f0e6c0708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 --2- 172.21.15.129:0/531387976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c152810 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 -- 172.21.15.129:0/531387976 >> 172.21.15.129:0/531387976 conn(0x7f0e6c0aa740 msgr2=0x7f0e6c0aab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 -- 172.21.15.129:0/531387976 shutdown_connections 2024-04-25T01:01:39.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.480+0000 7f0e743b0640 1 -- 172.21.15.129:0/531387976 wait complete. 2024-04-25T01:01:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.481+0000 7f0e743b0640 1 Processor -- start 2024-04-25T01:01:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.481+0000 7f0e743b0640 1 -- start start 2024-04-25T01:01:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e6c152d50 0x7f0e6c0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e6c1589c0 0x7f0e6c14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c0c6d30 con 0x7f0e6c158da0 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c0c6a50 con 0x7f0e6c1589c0 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6c0c27a0 con 0x7f0e6c152d50 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c14f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34376/0 (socket says 172.21.15.129:34376) 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 -- 172.21.15.129:0/654630206 learned_addr learned my addr 172.21.15.129:0/654630206 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7214c640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e6c152d50 0x7f0e6c0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7194b640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e6c1589c0 0x7f0e6c14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 -- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e6c152d50 msgr2=0x7f0e6c0a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e6c152d50 0x7f0e6c0a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 -- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e6c1589c0 msgr2=0x7f0e6c14efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e6c1589c0 0x7f0e6c14efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 -- 172.21.15.129:0/654630206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c1ede30 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e7294d640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c14f520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007e40 tx=0x7f0e6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64014620 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e64019070 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6c1ee110 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64014a20 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.482+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6c1ee6d0 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.483+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6c075550 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.484+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e64014bc0 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.484+0000 7f0e62ffd640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e5006be60 0x7f0e5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.484+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e6408b910 con 0x7f0e6c158da0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.484+0000 7f0e7214c640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e5006be60 0x7f0e5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.485+0000 7f0e7214c640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e5006be60 0x7f0e5006e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005880 tx=0x7f0e68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.489+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e640599c0 con 0x7f0e6c158da0 2024-04-25T01:01:39.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.698+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e6c068630 con 0x7f0e6c158da0 2024-04-25T01:01:39.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.699+0000 7f0e62ffd640 1 -- 172.21.15.129:0/654630206 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f0e6405dcc0 con 0x7f0e6c158da0 2024-04-25T01:01:39.699 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:39.699 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:37.763659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:01:39.700 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 7 2024-04-25T01:01:39.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e5006be60 msgr2=0x7f0e5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e5006be60 0x7f0e5006e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005880 tx=0x7f0e68009040 comp rx=0 tx=0).stop 2024-04-25T01:01:39.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 msgr2=0x7f0e6c14f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c14f520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e64007e40 tx=0x7f0e6400e4b0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 shutdown_connections 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e5006be60 0x7f0e5006e320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e6c158da0 0x7f0e6c14f520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e6c1589c0 0x7f0e6c14efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 --2- 172.21.15.129:0/654630206 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e6c152d50 0x7f0e6c0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 >> 172.21.15.129:0/654630206 conn(0x7f0e6c0aa740 msgr2=0x7f0e6c0aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 shutdown_connections 2024-04-25T01:01:39.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.701+0000 7f0e743b0640 1 -- 172.21.15.129:0/654630206 wait complete. 2024-04-25T01:01:39.712 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:39.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.807+0000 7f51ed08c640 1 Processor -- start 2024-04-25T01:01:39.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51ed08c640 1 -- start start 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51ed08c640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f51e814f260 0x7f51e80c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51ed08c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e80733a0 con 0x7f51e8155d70 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51ed08c640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e8073680 con 0x7f51e8067300 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51ed08c640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f51e8156180 con 0x7f51e814f260 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51e759e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f51e8155d70 0x7f51e80c1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60178/0 (socket says 172.21.15.129:60178) 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.808+0000 7f51e759e640 1 -- 172.21.15.129:0/3827691402 learned_addr learned my addr 172.21.15.129:0/3827691402 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041427437 0 0) 0x7f51e80733a0 con 0x7f51e8155d70 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51d4003520 con 0x7f51e8155d70 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827691402 >> v2:172.21.15.171:3300/0 conn(0x7f51e814f260 0x7f51e80c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1844439991 0 0) 0x7f51d4003520 con 0x7f51e8155d70 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 >> v2:172.21.15.171:3300/0 conn(0x7f51e814f260 msgr2=0x7f51e80c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 --2- 172.21.15.129:0/3827691402 >> v2:172.21.15.171:3300/0 conn(0x7f51e814f260 0x7f51e80c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 >> v1:172.21.15.171:6789/0 conn(0x7f51e8067300 legacy=0x7f51e81590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e81588e0 con 0x7f51e8155d70 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f51e00019f0 con 0x7f51e8155d70 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 >> v1:172.21.15.129:6789/0 conn(0x7f51e8155d70 legacy=0x7f51e80c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51d4004800 0x7f51d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d40051c0 0x7f51d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8156180 con 0x7f51d40039b0 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f51d400c160 con 0x7f51d40051c0 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f51d400c360 con 0x7f51d4004800 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e659c640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51d4004800 0x7f51d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e759e640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d40051c0 0x7f51d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51d4004800 msgr2=0x7f51d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51d4004800 0x7f51d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d40051c0 msgr2=0x7f51d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51d40051c0 0x7f51d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827691402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d4003520 con 0x7f51d40039b0 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.809+0000 7f51e759e640 1 -- 172.21.15.129:0/3827691402 reap_dead start 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.810+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51d4003dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009ac0 tx=0x7f51d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159d874cdbc5db15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.810+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0036070 con 0x7f51d40039b0 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.810+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d0031400 con 0x7f51d40039b0 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.810+0000 7f51e5d9b640 1 -- 172.21.15.129:0/3827691402 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0009690 con 0x7f51d40039b0 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.810+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827691402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 msgr2=0x7f51d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.810+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51d4003dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009ac0 tx=0x7f51d4005830 comp rx=0 tx=0).stop 2024-04-25T01:01:39.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.811+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827691402 shutdown_connections 2024-04-25T01:01:39.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.811+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827691402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51d4003dd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.811+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827691402 >> 172.21.15.129:0/3827691402 conn(0x7f51e80c71f0 msgr2=0x7f51e80c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:39.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.811+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827691402 shutdown_connections 2024-04-25T01:01:39.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.811+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827691402 wait complete. 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.812+0000 7f51ed08c640 1 Processor -- start 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.812+0000 7f51ed08c640 1 -- start start 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51e80c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51e81583c0 0x7f51e8147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51e8157920 0x7f51e8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f51e814f640 con 0x7f51d40039b0 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8158c50 con 0x7f51e81583c0 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f51e814f7c0 con 0x7f51e8157920 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51e80c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51e80c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34396/0 (socket says 172.21.15.129:34396) 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827097603 learned_addr learned my addr 172.21.15.129:0/3827097603 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e659c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51e81583c0 0x7f51e8147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e759e640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51e8157920 0x7f51e8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51e8157920 msgr2=0x7f51e8157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51e8157920 0x7f51e8157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51e81583c0 msgr2=0x7f51e8147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51e81583c0 0x7f51e8147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 -- 172.21.15.129:0/3827097603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e80c3ef0 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51e6d9d640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51e80c67c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51d002abb0 tx=0x7f51d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d003f020 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51d0037040 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e80c41d0 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0036070 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.813+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e80c6e60 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.814+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e8075550 con 0x7f51d40039b0 2024-04-25T01:01:39.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.815+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51d0042020 con 0x7f51d40039b0 2024-04-25T01:01:39.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.815+0000 7f51df7fe640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51c806be30 0x7f51c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:39.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.815+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f51d00ab2d0 con 0x7f51d40039b0 2024-04-25T01:01:39.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.815+0000 7f51e659c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51c806be30 0x7f51c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:39.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.816+0000 7f51e659c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51c806be30 0x7f51c806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f51e80aae20 tx=0x7f51d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:39.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.819+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f51d0049020 con 0x7f51d40039b0 2024-04-25T01:01:39.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.995+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51e80ac720 con 0x7f51d40039b0 2024-04-25T01:01:39.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.996+0000 7f51df7fe640 1 -- 172.21.15.129:0/3827097603 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f51d00794a0 con 0x7f51d40039b0 2024-04-25T01:01:39.997 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:39.997 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:39.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51c806be30 msgr2=0x7f51c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:39.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51c806be30 0x7f51c806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f51e80aae20 tx=0x7f51d8009040 comp rx=0 tx=0).stop 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 msgr2=0x7f51e80c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51e80c67c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51d002abb0 tx=0x7f51d0004730 comp rx=0 tx=0).stop 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 shutdown_connections 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51c806be30 0x7f51c806e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f51e8157920 0x7f51e8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51e81583c0 0x7f51e8147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 --2- 172.21.15.129:0/3827097603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51d40039b0 0x7f51e80c67c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 >> 172.21.15.129:0/3827097603 conn(0x7f51e80c71f0 msgr2=0x7f51e814c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 shutdown_connections 2024-04-25T01:01:40.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:39.998+0000 7f51ed08c640 1 -- 172.21.15.129:0/3827097603 wait complete. 2024-04-25T01:01:40.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:01:32.731107+0000', 'modified': '2024-04-25T01:01:37.763659+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.171:6837/1355919064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6835', 'nonce': 1355919064}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 1355919064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-04-25T01:01:40.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:01:40.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:01:40.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.521+0000 7f7d108d4640 1 Processor -- start 2024-04-25T01:01:40.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.521+0000 7f7d108d4640 1 -- start start 2024-04-25T01:01:40.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d108d4640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7d0c14f6f0 0x7f7d0c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d108d4640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7d0c0ac540 0x7f7d0c0c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d108d4640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c0c5180 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d108d4640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c155720 con 0x7f7d0c14f6f0 2024-04-25T01:01:40.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d108d4640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c153010 con 0x7f7d0c0ac540 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d0a59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7d0c14f6f0 0x7f7d0c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d0a59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7d0c14f6f0 0x7f7d0c14fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34408/0 (socket says 172.21.15.129:34408) 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.522+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4075619494 learned_addr learned my addr 172.21.15.129:0/4075619494 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.523+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4075619494 >> v2:172.21.15.171:3300/0 conn(0x7f7d0c0ac540 msgr2=0x7f7d0c0c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.523+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/4075619494 >> v2:172.21.15.171:3300/0 conn(0x7f7d0c0ac540 0x7f7d0c0c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.523+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4075619494 >> v1:172.21.15.171:6789/0 conn(0x7f7d0c06d6c0 legacy=0x7f7d0c0c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.523+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4075619494 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c1559e0 con 0x7f7d0c14f6f0 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.523+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/4075619494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c14fad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0c5f00 tx=0x7f7cf402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b348175db6d7dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.523+0000 7f7d0959a640 1 -- 172.21.15.129:0/4075619494 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4032710 con 0x7f7d0c14f6f0 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d0959a640 1 -- 172.21.15.129:0/4075619494 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf40328b0 con 0x7f7d0c14f6f0 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d0959a640 1 -- 172.21.15.129:0/4075619494 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4032ca0 con 0x7f7d0c14f6f0 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 -- 172.21.15.129:0/4075619494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c14f6f0 msgr2=0x7f7d0c14fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4075619494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c14fad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0c5f00 tx=0x7f7cf402c850 comp rx=0 tx=0).stop 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 -- 172.21.15.129:0/4075619494 shutdown_connections 2024-04-25T01:01:40.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4075619494 >> v2:172.21.15.171:3300/0 conn(0x7f7d0c0ac540 0x7f7d0c0c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4075619494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c14fad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 -- 172.21.15.129:0/4075619494 >> 172.21.15.129:0/4075619494 conn(0x7f7d0c0a86b0 msgr2=0x7f7d0c0a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 -- 172.21.15.129:0/4075619494 shutdown_connections 2024-04-25T01:01:40.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.524+0000 7f7d108d4640 1 -- 172.21.15.129:0/4075619494 wait complete. 2024-04-25T01:01:40.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.525+0000 7f7d108d4640 1 Processor -- start 2024-04-25T01:01:40.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 -- start start 2024-04-25T01:01:40.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 0x7f7d0c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7d0c0ac540 0x7f7d0c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c152c30 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c155fa0 con 0x7f7d0c14f6f0 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d108d4640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c152db0 con 0x7f7d0c0ac540 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d0a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 0x7f7d0c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d0a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 0x7f7d0c155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34422/0 (socket says 172.21.15.129:34422) 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.526+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4049739701 learned_addr learned my addr 172.21.15.129:0/4049739701 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7d0c0ac540 msgr2=0x7f7d0c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7d0c0ac540 0x7f7d0c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7d0c0ac540 0x7f7d0c152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d0c14f6f0 msgr2=0x7f7d0c153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0a59c640 1 -- 172.21.15.129:0/4049739701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c1db5f0 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 0x7f7d0c155010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402c890 tx=0x7f7cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d02ffd640 1 -- 172.21.15.129:0/4049739701 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4044070 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d02ffd640 1 -- 172.21.15.129:0/4049739701 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf4003810 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c145050 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d02ffd640 1 -- 172.21.15.129:0/4049739701 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4003c00 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.527+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c1db8b0 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.529+0000 7f7d02ffd640 1 -- 172.21.15.129:0/4049739701 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cf4047070 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.529+0000 7f7d02ffd640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7cec06be30 0x7f7cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.529+0000 7f7d02ffd640 1 -- 172.21.15.129:0/4049739701 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7cf40ae5d0 con 0x7f7d0c06d6c0 2024-04-25T01:01:40.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.529+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7cec06be30 0x7f7cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.530+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7cec06be30 0x7f7cec06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c1544b0 tx=0x7f7cfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.530+0000 7f7d135e9740 1 Processor -- start 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.530+0000 7f7d135e9740 1 -- start start 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf3df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 0x55e2cf3dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d135e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2cf307f00 con 0x55e2cf3a58e0 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e2cf2b0750 con 0x55e2cf3c6510 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x55e2cf303ec0 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 0x55e2cf3dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 0x55e2cf3dfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34438/0 (socket says 172.21.15.129:34438) 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 -- 172.21.15.129:0/393928980 learned_addr learned my addr 172.21.15.129:0/393928980 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf3df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 -- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 msgr2=0x55e2cf3df940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf3df940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 -- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 msgr2=0x55e2cf3e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 -- 172.21.15.129:0/393928980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cf01c2d0 con 0x55e2cf3a58e0 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.531+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 0x55e2cf3dfe80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00ca90 tx=0x7f7cfc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea64625d1f5dc872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7cebfff640 1 -- 172.21.15.129:0/393928980 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc023070 con 0x55e2cf3a58e0 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7cebfff640 1 -- 172.21.15.129:0/393928980 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc013ba0 con 0x55e2cf3a58e0 2024-04-25T01:01:40.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7cebfff640 1 -- 172.21.15.129:0/393928980 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc01e400 con 0x55e2cf3a58e0 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 -- 172.21.15.129:0/393928980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 msgr2=0x55e2cf3dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 0x55e2cf3dfe80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00ca90 tx=0x7f7cfc00f000 comp rx=0 tx=0).stop 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 -- 172.21.15.129:0/393928980 shutdown_connections 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3a58e0 0x55e2cf3dfe80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 --2- 172.21.15.129:0/393928980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf3df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 -- 172.21.15.129:0/393928980 >> 172.21.15.129:0/393928980 conn(0x55e2cf3a1640 msgr2=0x55e2cf3c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 -- 172.21.15.129:0/393928980 shutdown_connections 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.532+0000 7f7d135e9740 1 -- 172.21.15.129:0/393928980 wait complete. 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.533+0000 7f7d135e9740 1 -- start start 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.533+0000 7f7d135e9740 1 Processor -- start 2024-04-25T01:01:40.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf39f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf39f940 0x55e2cf3f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2cf3a5dc0 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e2cf2c3cf0 con 0x55e2cf3c6510 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x55e2cf3df4b0 con 0x55e2cf39f940 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d135e9740 10 client.0 _ll_register_callbacks cb 0x55e2cf2c36e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf39f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf39f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34452/0 (socket says 172.21.15.129:34452) 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 -- 172.21.15.129:0/689833749 learned_addr learned my addr 172.21.15.129:0/689833749 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf39f940 0x55e2cf3f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 -- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf39f940 msgr2=0x55e2cf3f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf39f940 0x55e2cf3f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 -- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 msgr2=0x55e2cf3c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 -- 172.21.15.129:0/689833749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2cf0e66b0 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf39f400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00afd0 tx=0x7f7cfc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d640ceeb190d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.534+0000 7f7ce97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7ce97fa640 1 -- 172.21.15.129:0/689833749 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc02f060 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7ce97fa640 1 -- 172.21.15.129:0/689833749 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7cfc01e810 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cfc013ba0 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7ce97fa640 1 -- 172.21.15.129:0/689833749 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f7cfc035050 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 10 client.4416 fetch_fsmap learned FSMap version 7 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55e2cf3de310 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7ce97fa640 1 -- 172.21.15.129:0/689833749 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc013ba0 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7ce97fa640 1 -- 172.21.15.129:0/689833749 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f7cfc038020 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 10 client.4416 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 10 client.4416 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 20 client.4416 populate_metadata read hostname 'smithi129' 2024-04-25T01:01:40.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf3f4920 0x55e2cf3f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 4 client.4416 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 --> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2cf315440 con 0x55e2cf3f4920 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.535+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf3f4920 0x55e2cf3f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.536+0000 7f7ce97fa640 10 client.4416 ms_handle_connect on v2:172.21.15.171:6835/1355919064 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.536+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf3f4920 0x55e2cf3f4d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f310af05ceb27a server_cookie=e12dd12b894aeef in_seq=0 out_seq=0 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.536+0000 7f7ce97fa640 1 -- 172.21.15.129:0/689833749 <== mds.0 v2:172.21.15.171:6835/1355919064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2cf315440 con 0x55e2cf3f4920 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.536+0000 7f7ce97fa640 10 client.4416 handle_command_reply: tid=0 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.536+0000 7f7d135e9740 1 client.4416 shutdown 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 msgr2=0x55e2cf39f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf39f400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00afd0 tx=0x7f7cfc013000 comp rx=0 tx=0).stop 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 shutdown_connections 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf3f4920 0x55e2cf3f4d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf39f940 0x55e2cf3f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf39f400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 --2- 172.21.15.129:0/689833749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7ce97fa640 0 client.4416 ms_handle_reset on v2:172.21.15.171:6835/1355919064 2024-04-25T01:01:40.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 >> 172.21.15.129:0/689833749 conn(0x55e2cf3a58e0 msgr2=0x55e2cf3e4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 shutdown_connections 2024-04-25T01:01:40.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 1 -- 172.21.15.129:0/689833749 wait complete. 2024-04-25T01:01:40.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.537+0000 7f7d135e9740 20 client.4416 trim_cache size 0 max 16384 2024-04-25T01:01:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.555+0000 7f7d135e9740 1 Processor -- start 2024-04-25T01:01:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.555+0000 7f7d135e9740 1 -- start start 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf39f940 0x55e2cf3d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf39e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d135e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1f3cd0 con 0x55e2cf3c6510 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1f3030 con 0x55e2cf39f940 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c1f3680 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34462/0 (socket says 172.21.15.129:34462) 2024-04-25T01:01:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 -- 172.21.15.129:0/1327358489 learned_addr learned my addr 172.21.15.129:0/1327358489 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf39f940 0x55e2cf3d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf39e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 -- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 msgr2=0x55e2cf39e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf39e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 -- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf39f940 msgr2=0x55e2cf3d6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf39f940 0x55e2cf3d6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 -- 172.21.15.129:0/1327358489 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cf3e0400 con 0x55e2cf3c6510 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf39f940 0x55e2cf3d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c48a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc008010 tx=0x7f7cfc025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d903230b5989e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7ceb7fe640 1 -- 172.21.15.129:0/1327358489 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc03e070 con 0x55e2cf3c6510 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7ceb7fe640 1 -- 172.21.15.129:0/1327358489 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc024040 con 0x55e2cf3c6510 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.556+0000 7f7ceb7fe640 1 -- 172.21.15.129:0/1327358489 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc023050 con 0x55e2cf3c6510 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 -- 172.21.15.129:0/1327358489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 msgr2=0x55e2cf3c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c48a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc008010 tx=0x7f7cfc025000 comp rx=0 tx=0).stop 2024-04-25T01:01:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 -- 172.21.15.129:0/1327358489 shutdown_connections 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3e2ad0 0x55e2cf39e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf3c6510 0x55e2cf3c48a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 --2- 172.21.15.129:0/1327358489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf39f940 0x55e2cf3d6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 -- 172.21.15.129:0/1327358489 >> 172.21.15.129:0/1327358489 conn(0x55e2cf3a58e0 msgr2=0x55e2cf3f0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 -- 172.21.15.129:0/1327358489 shutdown_connections 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 -- 172.21.15.129:0/1327358489 wait complete. 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.557+0000 7f7d135e9740 1 -- start start 2024-04-25T01:01:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 Processor -- start 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 0x55e2cf39e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3c6510 0x55e2cf3e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf45c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2cf3a0970 con 0x55e2cf39f940 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e2cf3a5dc0 con 0x55e2cf3e2ad0 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x55e2cf3c53d0 con 0x55e2cf3c6510 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d135e9740 10 client.0 _ll_register_callbacks cb 0x7f7cfc0139a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.558+0000 7f7d0a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 0x55e2cf39e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 0x55e2cf39e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34466/0 (socket says 172.21.15.129:34466) 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 -- 172.21.15.129:0/414688220 learned_addr learned my addr 172.21.15.129:0/414688220 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3c6510 0x55e2cf3e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0ad9d640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf45c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 -- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3c6510 msgr2=0x55e2cf3e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3c6510 0x55e2cf3e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 -- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3e2ad0 msgr2=0x55e2cf45c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf45c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 -- 172.21.15.129:0/414688220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2cf3a4bb0 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d0a59c640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 0x55e2cf39e650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402ce40 tx=0x7f7cf4042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f700c1f4c97c3f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7ce97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7ce97fa640 1 -- 172.21.15.129:0/414688220 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf404e040 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7ce97fa640 1 -- 172.21.15.129:0/414688220 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7cf407a9f0 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cfc024040 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7ce97fa640 1 -- 172.21.15.129:0/414688220 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f7cf40852d0 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 10 client.4422 fetch_fsmap learned FSMap version 7 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55e2cf3e1500 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7ce97fa640 1 -- 172.21.15.129:0/414688220 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc024040 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7ce97fa640 1 -- 172.21.15.129:0/414688220 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f7cf400e040 con 0x55e2cf39f940 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 10 client.4422 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 20 client.4422 populate_metadata read hostname 'smithi129' 2024-04-25T01:01:40.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf403930 0x55e2cf403d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 4 client.4422 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.559+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 --> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] -- command(tid 0: {"prefix": "status"}) -- 0x55e2cf3dc8a0 con 0x55e2cf403930 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf403930 0x55e2cf403d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d09d9b640 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf403930 0x55e2cf403d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86977cd26cd4232 server_cookie=dbd80e1f1bb3713 in_seq=0 out_seq=0 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7ce97fa640 10 client.4422 ms_handle_connect on v2:172.21.15.171:6835/1355919064 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7ce97fa640 1 -- 172.21.15.129:0/414688220 <== mds.0 v2:172.21.15.171:6835/1355919064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55e2cf3dc8a0 con 0x55e2cf403930 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7ce97fa640 10 client.4422 handle_command_reply: tid=0 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 client.4422 shutdown 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 msgr2=0x55e2cf39e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 0x55e2cf39e650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402ce40 tx=0x7f7cf4042040 comp rx=0 tx=0).stop 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 shutdown_connections 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x55e2cf403930 0x55e2cf403d10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x55e2cf3c6510 0x55e2cf3e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e2cf3e2ad0 0x55e2cf45c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 --2- 172.21.15.129:0/414688220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2cf39f940 0x55e2cf39e650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7ce97fa640 0 client.4422 ms_handle_reset on v2:172.21.15.171:6835/1355919064 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 >> 172.21.15.129:0/414688220 conn(0x55e2cf3a58e0 msgr2=0x55e2cf3a5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 shutdown_connections 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 1 -- 172.21.15.129:0/414688220 wait complete. 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.560+0000 7f7d135e9740 20 client.4422 trim_cache size 0 max 16384 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:01:40.568 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "4b43d0d1-0ff7-4061-ba99-dacfcc159ca4", 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4294, 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 7.1743050840000002, 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 7, 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 25, 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 14.320135512 2024-04-25T01:01:40.569 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7cec06be30 msgr2=0x7f7cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7cec06be30 0x7f7cec06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c1544b0 tx=0x7f7cfc008040 comp rx=0 tx=0).stop 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 msgr2=0x7f7d0c155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 0x7f7d0c155010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cf402c890 tx=0x7f7cf40046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 shutdown_connections 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7cec06be30 0x7f7cec06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7d0c0ac540 0x7f7d0c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d0c14f6f0 0x7f7d0c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 --2- 172.21.15.129:0/4049739701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d0c06d6c0 0x7f7d0c155010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 >> 172.21.15.129:0/4049739701 conn(0x7f7d0c0a86b0 msgr2=0x7f7d0c0a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 shutdown_connections 2024-04-25T01:01:40.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.561+0000 7f7d108d4640 1 -- 172.21.15.129:0/4049739701 wait complete. 2024-04-25T01:01:40.572 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:01:40.572 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:40.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:01:40.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.668+0000 7ff318c88640 1 Processor -- start 2024-04-25T01:01:40.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.668+0000 7ff318c88640 1 -- start start 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff318c88640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff3140c6190 0x7ff3140c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff318c88640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff31406d270 0x7ff3140abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff318c88640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3141530e0 con 0x7ff3140c2aa0 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff318c88640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff3140c2f60 con 0x7ff31406d270 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff318c88640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7ff314153360 con 0x7ff3140c6190 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff31406d270 0x7ff3140abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff31406d270 0x7ff3140abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34476/0 (socket says 172.21.15.129:34476) 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 -- 172.21.15.129:0/2136754353 learned_addr learned my addr 172.21.15.129:0/2136754353 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 -- 172.21.15.129:0/2136754353 >> v2:172.21.15.171:3300/0 conn(0x7ff3140c6190 msgr2=0x7ff3140c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 --2- 172.21.15.129:0/2136754353 >> v2:172.21.15.171:3300/0 conn(0x7ff3140c6190 0x7ff3140c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 -- 172.21.15.129:0/2136754353 >> v1:172.21.15.171:6789/0 conn(0x7ff3140c2aa0 legacy=0x7ff3141566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:40.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.669+0000 7ff311d9b640 1 -- 172.21.15.129:0/2136754353 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff314153640 con 0x7ff31406d270 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.670+0000 7ff311d9b640 1 --2- 172.21.15.129:0/2136754353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff31406d270 0x7ff3140abcc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3140c4430 tx=0x7ff30802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a0eb3a0ce5036e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.670+0000 7ff31159a640 1 -- 172.21.15.129:0/2136754353 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308037070 con 0x7ff31406d270 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.670+0000 7ff31159a640 1 -- 172.21.15.129:0/2136754353 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff308032440 con 0x7ff31406d270 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.670+0000 7ff31159a640 1 -- 172.21.15.129:0/2136754353 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308032810 con 0x7ff31406d270 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 -- 172.21.15.129:0/2136754353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff31406d270 msgr2=0x7ff3140abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 --2- 172.21.15.129:0/2136754353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff31406d270 0x7ff3140abcc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3140c4430 tx=0x7ff30802a230 comp rx=0 tx=0).stop 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 -- 172.21.15.129:0/2136754353 shutdown_connections 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 --2- 172.21.15.129:0/2136754353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff31406d270 0x7ff3140abcc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 --2- 172.21.15.129:0/2136754353 >> v2:172.21.15.171:3300/0 conn(0x7ff3140c6190 0x7ff3140c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 -- 172.21.15.129:0/2136754353 >> 172.21.15.129:0/2136754353 conn(0x7ff3140a86b0 msgr2=0x7ff3140a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 -- 172.21.15.129:0/2136754353 shutdown_connections 2024-04-25T01:01:40.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.671+0000 7ff318c88640 1 -- 172.21.15.129:0/2136754353 wait complete. 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.672+0000 7ff318c88640 1 Processor -- start 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.672+0000 7ff318c88640 1 -- start start 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff31406d270 0x7ff314154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3140c2aa0 0x7ff3140aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 0x7ff31414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff314156f60 con 0x7ff3140c6190 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff314156de0 con 0x7ff3140c2aa0 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff314153e40 con 0x7ff31406d270 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 0x7ff31414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 0x7ff31414b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34484/0 (socket says 172.21.15.129:34484) 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 -- 172.21.15.129:0/467068502 learned_addr learned my addr 172.21.15.129:0/467068502 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff31259c640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff31406d270 0x7ff314154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff311d9b640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3140c2aa0 0x7ff3140aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 -- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff31406d270 msgr2=0x7ff314154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff31406d270 0x7ff314154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 -- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3140c2aa0 msgr2=0x7ff3140aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3140c2aa0 0x7ff3140aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 -- 172.21.15.129:0/467068502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31414a3c0 con 0x7ff3140c6190 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff312d9d640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 0x7ff31414b180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c780 tx=0x7ff30c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c017070 con 0x7ff3140c6190 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c003da0 con 0x7ff3140c6190 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31414a6a0 con 0x7ff3140c6190 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0052b0 con 0x7ff3140c6190 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.673+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3140a5460 con 0x7ff3140c6190 2024-04-25T01:01:40.676 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.674+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31406cd20 con 0x7ff3140c6190 2024-04-25T01:01:40.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.675+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff30c01a070 con 0x7ff3140c6190 2024-04-25T01:01:40.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.675+0000 7ff302ffd640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff2f406be30 0x7ff2f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:40.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.675+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff30c08a700 con 0x7ff3140c6190 2024-04-25T01:01:40.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.675+0000 7ff31259c640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff2f406be30 0x7ff2f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:40.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.676+0000 7ff31259c640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff2f406be30 0x7ff2f406e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff3040098e0 tx=0x7ff304009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:40.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.679+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff30c0587b0 con 0x7ff3140c6190 2024-04-25T01:01:40.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.890+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff314075550 con 0x7ff3140c6190 2024-04-25T01:01:40.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.890+0000 7ff302ffd640 1 -- 172.21.15.129:0/467068502 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7ff30c05cab0 con 0x7ff3140c6190 2024-04-25T01:01:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff2f406be30 msgr2=0x7ff2f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff2f406be30 0x7ff2f406e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff3040098e0 tx=0x7ff304009060 comp rx=0 tx=0).stop 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 msgr2=0x7ff31414b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 0x7ff31414b180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c780 tx=0x7ff30c00cc40 comp rx=0 tx=0).stop 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 shutdown_connections 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff2f406be30 0x7ff2f406e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3140c6190 0x7ff31414b180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3140c2aa0 0x7ff3140aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 --2- 172.21.15.129:0/467068502 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff31406d270 0x7ff314154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:40.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 >> 172.21.15.129:0/467068502 conn(0x7ff3140a86b0 msgr2=0x7ff314150a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:40.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 shutdown_connections 2024-04-25T01:01:40.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:40.892+0000 7ff318c88640 1 -- 172.21.15.129:0/467068502 wait complete. 2024-04-25T01:01:40.903 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T01:01:40.903 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:01:40.903 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-04-25T01:01:40.903 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T01:01:40.903 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:40.904 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T01:01:40.904 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:01:40.904 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:01:40.904 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T01:01:40.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:40.904 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:01:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42718sec preferred_lft 42718sec 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::fb51:43e0:ef26:52be/64 scope link noprefixroute 2024-04-25T01:01:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:01:40.921 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:01:40.921 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:01:41.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:01:41.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:41.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:41.188 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:01:41.211 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-04-25T01:01:41.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:41.213 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-04-25T01:01:41.456 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-04-25T01:01:41.457 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:01:41.457 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:01:41.457 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:01:41.457 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:01:41.457 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:01:41.457 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:01:41.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:01:41.457 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:01:41.458 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:01:41.458 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:01:41.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:01:41.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:41.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:41.872 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T01:01:41.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:41.887 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-04-25T01:01:41.941 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:01:41.941 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:01:41.941 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:01:41.941 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T01:01:41.941 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:01:42.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:01:42.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:42.056 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T01:01:42.057 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:01:42.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:01:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:42.172 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:01:42.172 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:01:42.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-04-25T01:01:42.172 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:01:42.172 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:01:42.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:01:42.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:42.246 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T01:01:42.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:01:42.246 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:42.260 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T01:01:42.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:01:42.260 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:42.314 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T01:01:42.378 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:01:42.434 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T01:01:42.434 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T01:01:42.434 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T01:01:42.434 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T01:01:42.434 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T01:01:42.434 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:01:42.435 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:01:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:01:42.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:42.437 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T01:01:42.489 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:01:42.489 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T01:01:42.532 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T01:01:42.555 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T01:01:42.611 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:01:42.612 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:01:42.613 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:01:42.613 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:01:42.613 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T01:01:42.613 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:01:42.613 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:01:42.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:42.613 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T01:01:42.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:2024-04-25T01:01:42.619+0000 7f00a5464480 -1 init, newargv = 0x560a25fdd650 newargc=15 2024-04-25T01:01:42.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:ceph-fuse[41355]: starting ceph client 2024-04-25T01:01:42.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:ceph-fuse[41355]: starting fuse 2024-04-25T01:01:42.633 INFO:teuthology.orchestra.run.smithi129.stdout:43 2024-04-25T01:01:42.634 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> import glob 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> import re 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> import os 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> import subprocess 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> def _find_admin_socket(client_name): 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> files = glob.glob(asok_path) 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T01:01:42.634 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> # Given a non-glob path, it better be there 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> if "*" not in asok_path: 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> assert(len(files) == 1) 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> return files[0] 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> for f in files: 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> contents = proc_f.read() 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> if mountpoint in contents: 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> return f 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> print(_find_admin_socket("client.admin")) 2024-04-25T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi129:> ' 2024-04-25T01:01:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:/var/run/ceph/ceph-client.admin.41355.asok 2024-04-25T01:01:42.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:42.761 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41355.asok 2024-04-25T01:01:42.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:42.761 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41355.asok status 2024-04-25T01:01:42.894 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "metadata": { 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "entity_id": "admin", 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": "smithi129", 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "pid": "41355", 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "root": "/" 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_count": 0, 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_pinned_count": 0, 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4434, 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "inst": { 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "name": { 2024-04-25T01:01:42.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "client", 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "num": 4434 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1893294493 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 1893294493 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:01:42.896 INFO:teuthology.orchestra.run.smithi129.stdout: "inst_str": "client.4434 192.168.0.1:0/1893294493", 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "addr_str": "192.168.0.1:0/1893294493", 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "inode_count": 1, 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_epoch": 7, 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch": 25, 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklisted": false, 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs" 2024-04-25T01:01:42.897 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:01:42.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:42.902 DEBUG:teuthology.orchestra.run.smithi129:> stat --file-system '--printf=%T 2024-04-25T01:01:42.902 DEBUG:teuthology.orchestra.run.smithi129:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:42.917 INFO:teuthology.orchestra.run.smithi129.stdout:fuseblk 2024-04-25T01:01:42.917 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:42.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:42.917 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:42.981 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T01:01:43.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:01:43.085 INFO:teuthology.orchestra.run.smithi129.stdout:# file: . 2024-04-25T01:01:43.085 INFO:teuthology.orchestra.run.smithi129.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T01:01:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:43.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:01:43.091 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:43.091 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:01:43.106 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:43.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.242+0000 7f5df4638640 1 Processor -- start 2024-04-25T01:01:43.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.242+0000 7f5df4638640 1 -- start start 2024-04-25T01:01:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.243+0000 7f5df4638640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec0c0470 0x7f5dec0c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.243+0000 7f5df4638640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5dec06cb10 0x7f5dec06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.243+0000 7f5df4638640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dec06e780 con 0x7f5dec152bc0 2024-04-25T01:01:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.243+0000 7f5df4638640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f5dec06c680 con 0x7f5dec06cb10 2024-04-25T01:01:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.243+0000 7f5df4638640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec0c0970 con 0x7f5dec0c0470 2024-04-25T01:01:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.243+0000 7f5df2bd5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5dec06cb10 0x7f5dec06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df23d4640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f5dec152bc0 0x7f5dec159300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60186/0 (socket says 172.21.15.129:60186) 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df2bd5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5dec06cb10 0x7f5dec06d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34492/0 (socket says 172.21.15.129:34492) 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df23d4640 1 -- 172.21.15.129:0/4075416527 learned_addr learned my addr 172.21.15.129:0/4075416527 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df2bd5640 1 -- 172.21.15.129:0/4075416527 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec0c0470 msgr2=0x7f5dec0c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df1bd3640 1 --2- 172.21.15.129:0/4075416527 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec0c0470 0x7f5dec0c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df13d2640 1 -- 172.21.15.129:0/4075416527 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028210421 0 0) 0x7f5dec06e780 con 0x7f5dec152bc0 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df2bd5640 1 --2- 172.21.15.129:0/4075416527 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec0c0470 0x7f5dec0c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df2bd5640 1 -- 172.21.15.129:0/4075416527 >> v1:172.21.15.129:6789/0 conn(0x7f5dec152bc0 legacy=0x7f5dec159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.244+0000 7f5df2bd5640 1 -- 172.21.15.129:0/4075416527 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec0c0c50 con 0x7f5dec06cb10 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df2bd5640 1 --2- 172.21.15.129:0/4075416527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec06cb10 0x7f5dec06d6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009d90 tx=0x7f5de402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e76414b11abbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df13d2640 1 -- 172.21.15.129:0/4075416527 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4007740 con 0x7f5dec06cb10 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df13d2640 1 -- 172.21.15.129:0/4075416527 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de40078e0 con 0x7f5dec06cb10 2024-04-25T01:01:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df13d2640 1 -- 172.21.15.129:0/4075416527 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4007cb0 con 0x7f5dec06cb10 2024-04-25T01:01:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 -- 172.21.15.129:0/4075416527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec06cb10 msgr2=0x7f5dec06d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 --2- 172.21.15.129:0/4075416527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec06cb10 0x7f5dec06d6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009d90 tx=0x7f5de402a660 comp rx=0 tx=0).stop 2024-04-25T01:01:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 -- 172.21.15.129:0/4075416527 shutdown_connections 2024-04-25T01:01:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 --2- 172.21.15.129:0/4075416527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec06cb10 0x7f5dec06d6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 --2- 172.21.15.129:0/4075416527 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec0c0470 0x7f5dec0c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 -- 172.21.15.129:0/4075416527 >> 172.21.15.129:0/4075416527 conn(0x7f5dec0a86b0 msgr2=0x7f5dec0a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 -- 172.21.15.129:0/4075416527 shutdown_connections 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.245+0000 7f5df4638640 1 -- 172.21.15.129:0/4075416527 wait complete. 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.246+0000 7f5df4638640 1 Processor -- start 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.246+0000 7f5df4638640 1 -- start start 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df4638640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec06cb10 0x7f5dec071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df4638640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 0x7f5dec1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df4638640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dec152bc0 0x7f5dec1ee370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df4638640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec06cf70 con 0x7f5dec0c0470 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df4638640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec0c0f30 con 0x7f5dec152bc0 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df4638640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec06d0f0 con 0x7f5dec06cb10 2024-04-25T01:01:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 0x7f5dec1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 0x7f5dec1531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34504/0 (socket says 172.21.15.129:34504) 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 -- 172.21.15.129:0/4014530319 learned_addr learned my addr 172.21.15.129:0/4014530319 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df2bd5640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dec152bc0 0x7f5dec1ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df23d4640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec06cb10 0x7f5dec071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 -- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec06cb10 msgr2=0x7f5dec071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec06cb10 0x7f5dec071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 -- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dec152bc0 msgr2=0x7f5dec1ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dec152bc0 0x7f5dec1ee370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df1bd3640 1 -- 172.21.15.129:0/4014530319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec154e80 con 0x7f5dec0c0470 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df2bd5640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dec152bc0 0x7f5dec1ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.247+0000 7f5df23d4640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec06cb10 0x7f5dec071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.248+0000 7f5df1bd3640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 0x7f5dec1531d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c7f0 tx=0x7f5de800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.248+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8015070 con 0x7f5dec0c0470 2024-04-25T01:01:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.248+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de8004240 con 0x7f5dec0c0470 2024-04-25T01:01:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.248+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec071620 con 0x7f5dec0c0470 2024-04-25T01:01:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.248+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8007620 con 0x7f5dec0c0470 2024-04-25T01:01:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.248+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec071a00 con 0x7f5dec0c0470 2024-04-25T01:01:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.249+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec075550 con 0x7f5dec0c0470 2024-04-25T01:01:43.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.249+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5de80079d0 con 0x7f5dec0c0470 2024-04-25T01:01:43.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.250+0000 7f5de2ffd640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5dd006be30 0x7f5dd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.250+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5de808b510 con 0x7f5dec0c0470 2024-04-25T01:01:43.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.250+0000 7f5df23d4640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5dd006be30 0x7f5dd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.250+0000 7f5df23d4640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5dd006be30 0x7f5dd006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000f60 tx=0x7f5ddc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.254+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5de8059c10 con 0x7f5dec0c0470 2024-04-25T01:01:43.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.429+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5dec0bfd30 con 0x7f5dec0c0470 2024-04-25T01:01:43.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.430+0000 7f5de2ffd640 1 -- 172.21.15.129:0/4014530319 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f5de8059510 con 0x7f5dec0c0470 2024-04-25T01:01:43.431 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:43.431 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:43.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.432+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5dd006be30 msgr2=0x7f5dd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.432+0000 7f5df4638640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5dd006be30 0x7f5dd006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000f60 tx=0x7f5ddc009040 comp rx=0 tx=0).stop 2024-04-25T01:01:43.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.432+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 msgr2=0x7f5dec1531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.432+0000 7f5df4638640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 0x7f5dec1531d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c7f0 tx=0x7f5de800ccb0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 shutdown_connections 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5dd006be30 0x7f5dd006e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dec152bc0 0x7f5dec1ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dec0c0470 0x7f5dec1531d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 --2- 172.21.15.129:0/4014530319 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dec06cb10 0x7f5dec071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 >> 172.21.15.129:0/4014530319 conn(0x7f5dec0a86b0 msgr2=0x7f5dec151330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 shutdown_connections 2024-04-25T01:01:43.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.433+0000 7f5df4638640 1 -- 172.21.15.129:0/4014530319 wait complete. 2024-04-25T01:01:43.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:01:43.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:01:43.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.545+0000 7f29a2493640 1 Processor -- start 2024-04-25T01:01:43.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.545+0000 7f29a2493640 1 -- start start 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.545+0000 7f29a2493640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f299c14f6f0 0x7f299c0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.545+0000 7f29a2493640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0a6430 con 0x7f299c0c53d0 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.545+0000 7f29a2493640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0c4930 con 0x7f299c14fad0 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.545+0000 7f29a2493640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f299c0ab270 con 0x7f299c14f6f0 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f29a0a30640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f299c0c53d0 0x7f299c06d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60202/0 (socket says 172.21.15.129:60202) 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f29a0a30640 1 -- 172.21.15.129:0/2859056619 learned_addr learned my addr 172.21.15.129:0/2859056619 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299bfff640 1 --2- 172.21.15.129:0/2859056619 >> v2:172.21.15.171:3300/0 conn(0x7f299c14f6f0 0x7f299c0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 254532145 0 0) 0x7f299c0a6430 con 0x7f299c0c53d0 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2984003520 con 0x7f299c0c53d0 2024-04-25T01:01:43.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134137545 0 0) 0x7f299c0c4930 con 0x7f299c14fad0 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f299c0a6430 con 0x7f299c14fad0 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2825603684 0 0) 0x7f2984003520 con 0x7f299c0c53d0 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 >> v2:172.21.15.171:3300/0 conn(0x7f299c14f6f0 msgr2=0x7f299c0aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 --2- 172.21.15.129:0/2859056619 >> v2:172.21.15.171:3300/0 conn(0x7f299c14f6f0 0x7f299c0aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 >> v1:172.21.15.171:6789/0 conn(0x7f299c14fad0 legacy=0x7f299c152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0c5e20 con 0x7f299c0c53d0 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f29940019f0 con 0x7f299c0c53d0 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 >> v1:172.21.15.129:6789/0 conn(0x7f299c0c53d0 legacy=0x7f299c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2984003b40 0x7f2984003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f2984004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2984005280 0x7f298400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f298400c160 con 0x7f2984004800 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f298400c310 con 0x7f2984005280 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f298400c510 con 0x7f2984003b40 2024-04-25T01:01:43.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299b7fe640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f2984004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299bfff640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2984003b40 0x7f2984003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f29a0a30640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2984005280 0x7f298400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299b7fe640 1 -- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2984003b40 msgr2=0x7f2984003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299b7fe640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2984003b40 0x7f2984003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299b7fe640 1 -- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2984005280 msgr2=0x7f298400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299b7fe640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2984005280 0x7f298400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f299b7fe640 1 -- 172.21.15.129:0/2859056619 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2984003520 con 0x7f2984004800 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.546+0000 7f29a0a30640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2984005280 0x7f298400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.547+0000 7f29a0a30640 1 -- 172.21.15.129:0/2859056619 reap_dead start 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.547+0000 7f299b7fe640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f2984004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29900018a0 tx=0x7f2984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462210b1b464f658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.547+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900014f0 con 0x7f2984004800 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.547+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29900052d0 con 0x7f2984004800 2024-04-25T01:01:43.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.547+0000 7f299affd640 1 -- 172.21.15.129:0/2859056619 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299002ac40 con 0x7f2984004800 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 -- 172.21.15.129:0/2859056619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 msgr2=0x7f2984004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f2984004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29900018a0 tx=0x7f2984005830 comp rx=0 tx=0).stop 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 -- 172.21.15.129:0/2859056619 shutdown_connections 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 --2- 172.21.15.129:0/2859056619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f2984004c80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 -- 172.21.15.129:0/2859056619 >> 172.21.15.129:0/2859056619 conn(0x7f299c0a86b0 msgr2=0x7f299c0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 -- 172.21.15.129:0/2859056619 shutdown_connections 2024-04-25T01:01:43.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.548+0000 7f29a2493640 1 -- 172.21.15.129:0/2859056619 wait complete. 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.549+0000 7f29a2493640 1 Processor -- start 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.549+0000 7f29a2493640 1 -- start start 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a2493640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f299c145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a2493640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f299c14db90 0x7f299c14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a2493640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c1e6430 0x7f299c1e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a2493640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0c4eb0 con 0x7f2984004800 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a2493640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0c4bd0 con 0x7f299c1e6430 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a2493640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0c5860 con 0x7f299c14db90 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f299c145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f299c145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34524/0 (socket says 172.21.15.129:34524) 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 -- 172.21.15.129:0/3615615575 learned_addr learned my addr 172.21.15.129:0/3615615575 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:43.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299b7fe640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f299c14db90 0x7f299c14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f29a0a30640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c1e6430 0x7f299c1e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 -- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f299c14db90 msgr2=0x7f299c14dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f299c14db90 0x7f299c14dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 -- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c1e6430 msgr2=0x7f299c1e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c1e6430 0x7f299c1e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 -- 172.21.15.129:0/3615615575 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c071750 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f299bfff640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f299c145590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f298c011aa0 tx=0x7f298c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.550+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c00fb00 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.551+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298c00fca0 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.551+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c014660 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.551+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c0719c0 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.551+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c071f80 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.552+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c1e6070 con 0x7f2984004800 2024-04-25T01:01:43.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.552+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f298c012040 con 0x7f2984004800 2024-04-25T01:01:43.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.552+0000 7f2998ff9640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f298806bdd0 0x7f298806e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.552+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f298c08b5f0 con 0x7f2984004800 2024-04-25T01:01:43.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.553+0000 7f299b7fe640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f298806bdd0 0x7f298806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.553+0000 7f299b7fe640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f298806bdd0 0x7f298806e290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2990032000 tx=0x7f2990009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.557+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f298c019030 con 0x7f2984004800 2024-04-25T01:01:43.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.729+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f299c075550 con 0x7f2984004800 2024-04-25T01:01:43.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.729+0000 7f2998ff9640 1 -- 172.21.15.129:0/3615615575 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f298c0596a0 con 0x7f2984004800 2024-04-25T01:01:43.730 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f298806bdd0 msgr2=0x7f298806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f298806bdd0 0x7f298806e290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2990032000 tx=0x7f2990009030 comp rx=0 tx=0).stop 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 msgr2=0x7f299c145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f299c145590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f298c011aa0 tx=0x7f298c011f60 comp rx=0 tx=0).stop 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 shutdown_connections 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f298806bdd0 0x7f298806e290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f299c14db90 0x7f299c14dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c1e6430 0x7f299c1e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 --2- 172.21.15.129:0/3615615575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2984004800 0x7f299c145590 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 >> 172.21.15.129:0/3615615575 conn(0x7f299c0a86b0 msgr2=0x7f299c14ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:43.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 shutdown_connections 2024-04-25T01:01:43.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.731+0000 7f29a2493640 1 -- 172.21.15.129:0/3615615575 wait complete. 2024-04-25T01:01:43.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:01:43.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.842+0000 7f3c2222a640 1 Processor -- start 2024-04-25T01:01:43.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.842+0000 7f3c2222a640 1 -- start start 2024-04-25T01:01:43.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c2222a640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3c1c150750 0x7f3c1c0abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c2222a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3c1c0a5ce0 0x7f3c1c15db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c2222a640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c0a61e0 con 0x7f3c1c153e90 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c2222a640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1c1533b0 con 0x7f3c1c150750 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c2222a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1c153690 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3c1c150750 0x7f3c1c0abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3c1c150750 0x7f3c1c0abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34532/0 (socket says 172.21.15.129:34532) 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/4240623015 learned_addr learned my addr 172.21.15.129:0/4240623015 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1bfff640 1 --2- 172.21.15.129:0/4240623015 >> v2:172.21.15.171:3300/0 conn(0x7f3c1c0a5ce0 0x7f3c1c15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/4240623015 >> v2:172.21.15.171:3300/0 conn(0x7f3c1c0a5ce0 msgr2=0x7f3c1c15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 --2- 172.21.15.129:0/4240623015 >> v2:172.21.15.171:3300/0 conn(0x7f3c1c0a5ce0 0x7f3c1c15db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/4240623015 >> v1:172.21.15.171:6789/0 conn(0x7f3c1c153e90 legacy=0x7f3c1c156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.843+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/4240623015 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c153970 con 0x7f3c1c150750 2024-04-25T01:01:43.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.844+0000 7f3c1b7fe640 1 --2- 172.21.15.129:0/4240623015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c150750 0x7f3c1c0abcc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0a9480 tx=0x7f3c0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b29612e06d296d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.844+0000 7f3c1a7fc640 1 -- 172.21.15.129:0/4240623015 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c037070 con 0x7f3c1c150750 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.844+0000 7f3c1a7fc640 1 -- 172.21.15.129:0/4240623015 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c032440 con 0x7f3c1c150750 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.844+0000 7f3c1a7fc640 1 -- 172.21.15.129:0/4240623015 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c032810 con 0x7f3c1c150750 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 -- 172.21.15.129:0/4240623015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c150750 msgr2=0x7f3c1c0abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 --2- 172.21.15.129:0/4240623015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c150750 0x7f3c1c0abcc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0a9480 tx=0x7f3c0c02a230 comp rx=0 tx=0).stop 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 -- 172.21.15.129:0/4240623015 shutdown_connections 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 --2- 172.21.15.129:0/4240623015 >> v2:172.21.15.171:3300/0 conn(0x7f3c1c0a5ce0 0x7f3c1c15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 --2- 172.21.15.129:0/4240623015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c150750 0x7f3c1c0abcc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 -- 172.21.15.129:0/4240623015 >> 172.21.15.129:0/4240623015 conn(0x7f3c1c0a86b0 msgr2=0x7f3c1c0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 -- 172.21.15.129:0/4240623015 shutdown_connections 2024-04-25T01:01:43.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.845+0000 7f3c2222a640 1 -- 172.21.15.129:0/4240623015 wait complete. 2024-04-25T01:01:43.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.846+0000 7f3c2222a640 1 Processor -- start 2024-04-25T01:01:43.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.846+0000 7f3c2222a640 1 -- start start 2024-04-25T01:01:43.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.846+0000 7f3c2222a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 0x7f3c1c1460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c1c150750 0x7f3c1c0bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c1c153e90 0x7f3c1c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c06c3f0 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c06c270 con 0x7f3c1c150750 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c156ea0 con 0x7f3c1c153e90 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 0x7f3c1c1460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 0x7f3c1c1460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34534/0 (socket says 172.21.15.129:34534) 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/3150493956 learned_addr learned my addr 172.21.15.129:0/3150493956 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1bfff640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c1c153e90 0x7f3c1c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c1c153e90 msgr2=0x7f3c1c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1affd640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c1c150750 0x7f3c1c0bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c1c153e90 0x7f3c1c151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.851 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c1c150750 msgr2=0x7f3c1c0bfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c1c150750 0x7f3c1c0bfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 -- 172.21.15.129:0/3150493956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0a9f30 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1affd640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c1c150750 0x7f3c1c0bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c1b7fe640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 0x7f3c1c1460d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c02a270 tx=0x7f3c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c037040 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c042070 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c0a9c50 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c030bb0 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.847+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c14dba0 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.848+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c075550 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.849+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c0c047460 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.849+0000 7f3c03fff640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c0806be30 0x7f3c0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:43.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.849+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c0c0abe80 con 0x7f3c1c0a5ce0 2024-04-25T01:01:43.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.850+0000 7f3c1affd640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c0806be30 0x7f3c0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:43.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.850+0000 7f3c1affd640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c0806be30 0x7f3c0806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c156840 tx=0x7f3c10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:43.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:43.854+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c0c045360 con 0x7f3c1c0a5ce0 2024-04-25T01:01:44.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.025+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c1c1eb5b0 con 0x7f3c1c0a5ce0 2024-04-25T01:01:44.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.025+0000 7f3c03fff640 1 -- 172.21.15.129:0/3150493956 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f3c1c1eb5b0 con 0x7f3c1c0a5ce0 2024-04-25T01:01:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:01:44.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c0806be30 msgr2=0x7f3c0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c0806be30 0x7f3c0806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c156840 tx=0x7f3c10008040 comp rx=0 tx=0).stop 2024-04-25T01:01:44.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 msgr2=0x7f3c1c1460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 0x7f3c1c1460d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c02a270 tx=0x7f3c0c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 shutdown_connections 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c0806be30 0x7f3c0806e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c1c153e90 0x7f3c1c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c1c150750 0x7f3c1c0bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 --2- 172.21.15.129:0/3150493956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c1c0a5ce0 0x7f3c1c1460d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 >> 172.21.15.129:0/3150493956 conn(0x7f3c1c0a86b0 msgr2=0x7f3c1c14ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 shutdown_connections 2024-04-25T01:01:44.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.027+0000 7f3c2222a640 1 -- 172.21.15.129:0/3150493956 wait complete. 2024-04-25T01:01:44.039 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:01:44.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.137+0000 7fc3cc35c640 1 Processor -- start 2024-04-25T01:01:44.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.137+0000 7fc3cc35c640 1 -- start start 2024-04-25T01:01:44.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3cc35c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c4150750 0x7fc3c40abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3cc35c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c40a61e0 con 0x7fc3c40a5ce0 2024-04-25T01:01:44.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3cc35c640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c41533b0 con 0x7fc3c4153e90 2024-04-25T01:01:44.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3cc35c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c4153690 con 0x7fc3c4150750 2024-04-25T01:01:44.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3ca8f9640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc3c40a5ce0 0x7fc3c415db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60214/0 (socket says 172.21.15.129:60214) 2024-04-25T01:01:44.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3ca8f9640 1 -- 172.21.15.129:0/4113125464 learned_addr learned my addr 172.21.15.129:0/4113125464 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3ca0f8640 1 --2- 172.21.15.129:0/4113125464 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c4150750 0x7fc3c40abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3c90f6640 1 -- 172.21.15.129:0/4113125464 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693289906 0 0) 0x7fc3c40a61e0 con 0x7fc3c40a5ce0 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3c90f6640 1 -- 172.21.15.129:0/4113125464 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3b0003520 con 0x7fc3c40a5ce0 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.138+0000 7fc3c90f6640 1 -- 172.21.15.129:0/4113125464 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660026914 0 0) 0x7fc3c41533b0 con 0x7fc3c4153e90 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.139+0000 7fc3ca0f8640 1 -- 172.21.15.129:0/4113125464 >> v1:172.21.15.171:6789/0 conn(0x7fc3c4153e90 legacy=0x7fc3c4156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.139+0000 7fc3ca0f8640 1 -- 172.21.15.129:0/4113125464 >> v1:172.21.15.129:6789/0 conn(0x7fc3c40a5ce0 legacy=0x7fc3c415db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.139+0000 7fc3ca0f8640 1 -- 172.21.15.129:0/4113125464 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c4153970 con 0x7fc3c4150750 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.139+0000 7fc3ca0f8640 1 --2- 172.21.15.129:0/4113125464 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c4150750 0x7fc3c40abcc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00097d0 tx=0x7fc3c002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff786b9b77e385a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.139+0000 7fc3c90f6640 1 -- 172.21.15.129:0/4113125464 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0037070 con 0x7fc3c4150750 2024-04-25T01:01:44.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.139+0000 7fc3c90f6640 1 -- 172.21.15.129:0/4113125464 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c0032440 con 0x7fc3c4150750 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3c90f6640 1 -- 172.21.15.129:0/4113125464 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0032810 con 0x7fc3c4150750 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 -- 172.21.15.129:0/4113125464 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c4150750 msgr2=0x7fc3c40abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/4113125464 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c4150750 0x7fc3c40abcc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00097d0 tx=0x7fc3c002a210 comp rx=0 tx=0).stop 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 -- 172.21.15.129:0/4113125464 shutdown_connections 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/4113125464 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c4150750 0x7fc3c40abcc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 -- 172.21.15.129:0/4113125464 >> 172.21.15.129:0/4113125464 conn(0x7fc3c40a86b0 msgr2=0x7fc3c40a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 -- 172.21.15.129:0/4113125464 shutdown_connections 2024-04-25T01:01:44.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.140+0000 7fc3cc35c640 1 -- 172.21.15.129:0/4113125464 wait complete. 2024-04-25T01:01:44.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.141+0000 7fc3cc35c640 1 Processor -- start 2024-04-25T01:01:44.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.141+0000 7fc3cc35c640 1 -- start start 2024-04-25T01:01:44.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3cc35c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c40a5ce0 0x7fc3c4147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3cc35c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 0x7fc3c4148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3cc35c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3c4153e90 0x7fc3c40a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3cc35c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c406c3f0 con 0x7fc3c4150750 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3cc35c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c406c270 con 0x7fc3c4153e90 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3cc35c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c4156ea0 con 0x7fc3c40a5ce0 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 0x7fc3c4148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 0x7fc3c4148000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34536/0 (socket says 172.21.15.129:34536) 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 -- 172.21.15.129:0/2405687874 learned_addr learned my addr 172.21.15.129:0/2405687874 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3ca0f8640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c40a5ce0 0x7fc3c4147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3ca8f9640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3c4153e90 0x7fc3c40a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 -- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c40a5ce0 msgr2=0x7fc3c4147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c40a5ce0 0x7fc3c4147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 -- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3c4153e90 msgr2=0x7fc3c40a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3c4153e90 0x7fc3c40a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 -- 172.21.15.129:0/2405687874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c41e9a40 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3ca0f8640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c40a5ce0 0x7fc3c4147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3ca8f9640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3c4153e90 0x7fc3c40a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.142+0000 7fc3c98f7640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 0x7fc3c4148000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c410 tx=0x7fc3b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.143+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4017070 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.143+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b40035f0 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.143+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c40a66e0 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.143+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40039e0 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.143+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c40a6980 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.144+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c4075550 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.144+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3b401a070 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.144+0000 7fc3baffd640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3a806be60 0x7fc3a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.145+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc3b408a550 con 0x7fc3c4150750 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.145+0000 7fc3ca0f8640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3a806be60 0x7fc3a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.145+0000 7fc3ca0f8640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3a806be60 0x7fc3a806e320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0034000 tx=0x7fc3c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.149+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3b4058c50 con 0x7fc3c4150750 2024-04-25T01:01:44.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.322+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3c406c590 con 0x7fc3c4150750 2024-04-25T01:01:44.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.322+0000 7fc3baffd640 1 -- 172.21.15.129:0/2405687874 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc3b4058550 con 0x7fc3c4150750 2024-04-25T01:01:44.326 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:01:44.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3a806be60 msgr2=0x7fc3a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3a806be60 0x7fc3a806e320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0034000 tx=0x7fc3c002e000 comp rx=0 tx=0).stop 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 msgr2=0x7fc3c4148000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 0x7fc3c4148000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c410 tx=0x7fc3b400c8d0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 shutdown_connections 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3a806be60 0x7fc3a806e320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3c4153e90 0x7fc3c40a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3c4150750 0x7fc3c4148000 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 --2- 172.21.15.129:0/2405687874 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3c40a5ce0 0x7fc3c4147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 >> 172.21.15.129:0/2405687874 conn(0x7fc3c40a86b0 msgr2=0x7fc3c40c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 shutdown_connections 2024-04-25T01:01:44.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.324+0000 7fc3cc35c640 1 -- 172.21.15.129:0/2405687874 wait complete. 2024-04-25T01:01:44.336 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:01:44.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.434+0000 7fc2b45d5640 1 Processor -- start 2024-04-25T01:01:44.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.434+0000 7fc2b45d5640 1 -- start start 2024-04-25T01:01:44.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b45d5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc2ac0aad30 0x7fc2ac06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b45d5640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac0a6430 con 0x7fc2ac14fad0 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b45d5640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac0c4930 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b45d5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac0c5e20 con 0x7fc2ac0aad30 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b1b70640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc2ac14fad0 0x7fc2ac0c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60228/0 (socket says 172.21.15.129:60228) 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b1b70640 1 -- 172.21.15.129:0/1672127081 learned_addr learned my addr 172.21.15.129:0/1672127081 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b2b72640 1 --2- 172.21.15.129:0/1672127081 >> v2:172.21.15.171:3300/0 conn(0x7fc2ac0aad30 0x7fc2ac06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.435+0000 7fc2b136f640 1 -- 172.21.15.129:0/1672127081 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387745590 0 0) 0x7fc2ac0a6430 con 0x7fc2ac14fad0 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.436+0000 7fc2b136f640 1 -- 172.21.15.129:0/1672127081 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc298005730 con 0x7fc2ac14fad0 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.436+0000 7fc2b2b72640 1 -- 172.21.15.129:0/1672127081 >> v1:172.21.15.171:6789/0 conn(0x7fc2ac14f6f0 legacy=0x7fc2ac152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.436+0000 7fc2b2b72640 1 -- 172.21.15.129:0/1672127081 >> v1:172.21.15.129:6789/0 conn(0x7fc2ac14fad0 legacy=0x7fc2ac0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.436+0000 7fc2b2b72640 1 -- 172.21.15.129:0/1672127081 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac0a6150 con 0x7fc2ac0aad30 2024-04-25T01:01:44.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.436+0000 7fc2b2b72640 1 --2- 172.21.15.129:0/1672127081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac06d6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0c4b00 tx=0x7fc2a402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e36a39eb186099b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.439+0000 7fc2b136f640 1 -- 172.21.15.129:0/1672127081 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4031600 con 0x7fc2ac0aad30 2024-04-25T01:01:44.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.439+0000 7fc2b136f640 1 -- 172.21.15.129:0/1672127081 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a40317a0 con 0x7fc2ac0aad30 2024-04-25T01:01:44.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.439+0000 7fc2b136f640 1 -- 172.21.15.129:0/1672127081 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4031b70 con 0x7fc2ac0aad30 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.439+0000 7fc2b45d5640 1 -- 172.21.15.129:0/1672127081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 msgr2=0x7fc2ac06d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.439+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/1672127081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac06d6c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0c4b00 tx=0x7fc2a402a5d0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.440+0000 7fc2b45d5640 1 -- 172.21.15.129:0/1672127081 shutdown_connections 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.440+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/1672127081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac06d6c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.440+0000 7fc2b45d5640 1 -- 172.21.15.129:0/1672127081 >> 172.21.15.129:0/1672127081 conn(0x7fc2ac0a86b0 msgr2=0x7fc2ac0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.440+0000 7fc2b45d5640 1 -- 172.21.15.129:0/1672127081 shutdown_connections 2024-04-25T01:01:44.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.440+0000 7fc2b45d5640 1 -- 172.21.15.129:0/1672127081 wait complete. 2024-04-25T01:01:44.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.441+0000 7fc2b45d5640 1 Processor -- start 2024-04-25T01:01:44.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.441+0000 7fc2b45d5640 1 -- start start 2024-04-25T01:01:44.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.441+0000 7fc2b45d5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac1eed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.441+0000 7fc2b45d5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 0x7fc2ac070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b45d5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2ac14fad0 0x7fc2ac14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b45d5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0ab1a0 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b45d5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0c4eb0 con 0x7fc2ac0aad30 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b45d5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac0a5ce0 con 0x7fc2ac14fad0 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 0x7fc2ac070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 0x7fc2ac070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34544/0 (socket says 172.21.15.129:34544) 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 -- 172.21.15.129:0/2649978078 learned_addr learned my addr 172.21.15.129:0/2649978078 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b2371640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac1eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b2b72640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2ac14fad0 0x7fc2ac14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 -- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2ac14fad0 msgr2=0x7fc2ac14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2ac14fad0 0x7fc2ac14ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 -- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 msgr2=0x7fc2ac1eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac1eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 -- 172.21.15.129:0/2649978078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac155730 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b1b70640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 0x7fc2ac070940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c410 tx=0x7fc29c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c017070 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29c005ce0 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac155a10 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0072f0 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.442+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac14f460 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.443+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac075550 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.444+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc29c01a070 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.444+0000 7fc2a2ffd640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc29006be60 0x7fc29006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.444+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc29c08a420 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.444+0000 7fc2b2371640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc29006be60 0x7fc29006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.445+0000 7fc2b2371640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc29006be60 0x7fc29006e320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8001780 tx=0x7fc2a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.448+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc29c058590 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.618+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2ac0a81a0 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.619+0000 7fc2a2ffd640 1 -- 172.21.15.129:0/2649978078 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fc29c05c890 con 0x7fc2ac14f6f0 2024-04-25T01:01:44.620 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc29006be60 msgr2=0x7fc29006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc29006be60 0x7fc29006e320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8001780 tx=0x7fc2a8009040 comp rx=0 tx=0).stop 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 msgr2=0x7fc2ac070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 0x7fc2ac070940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c410 tx=0x7fc29c00c8d0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 shutdown_connections 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc29006be60 0x7fc29006e320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2ac14fad0 0x7fc2ac14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2ac14f6f0 0x7fc2ac070940 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 --2- 172.21.15.129:0/2649978078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2ac0aad30 0x7fc2ac1eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 >> 172.21.15.129:0/2649978078 conn(0x7fc2ac0a86b0 msgr2=0x7fc2ac0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 shutdown_connections 2024-04-25T01:01:44.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.621+0000 7fc2b45d5640 1 -- 172.21.15.129:0/2649978078 wait complete. 2024-04-25T01:01:44.633 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:01:44.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.730+0000 7f2977b63640 1 Processor -- start 2024-04-25T01:01:44.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f2977b63640 1 -- start start 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f2977b63640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f29700c9630 0x7f2970070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f2977b63640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f29700c8870 0x7f2970156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f2977b63640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2970070600 con 0x7f29700c75e0 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f2977b63640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f29700708e0 con 0x7f29700c9630 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f2977b63640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f2970070bc0 con 0x7f29700c8870 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f29758ff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f29700c9630 0x7f2970070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f29750fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f29700c75e0 0x7f29700aa6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60240/0 (socket says 172.21.15.129:60240) 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.731+0000 7f29750fe640 1 -- 172.21.15.129:0/1259556512 learned_addr learned my addr 172.21.15.129:0/1259556512 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:44.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f2976100640 1 --2- 172.21.15.129:0/1259556512 >> v2:172.21.15.171:3300/0 conn(0x7f29700c8870 0x7f2970156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f29758ff640 1 -- 172.21.15.129:0/1259556512 >> v2:172.21.15.171:3300/0 conn(0x7f29700c8870 msgr2=0x7f2970156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f29758ff640 1 --2- 172.21.15.129:0/1259556512 >> v2:172.21.15.171:3300/0 conn(0x7f29700c8870 0x7f2970156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f29758ff640 1 -- 172.21.15.129:0/1259556512 >> v1:172.21.15.129:6789/0 conn(0x7f29700c75e0 legacy=0x7f29700aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f29748fd640 1 -- 172.21.15.129:0/1259556512 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721128111 0 0) 0x7f2970070600 con 0x7f29700c75e0 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f29758ff640 1 -- 172.21.15.129:0/1259556512 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2970156a50 con 0x7f29700c9630 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f2976100640 1 --2- 172.21.15.129:0/1259556512 >> v2:172.21.15.171:3300/0 conn(0x7f29700c8870 0x7f2970156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.732+0000 7f29758ff640 1 --2- 172.21.15.129:0/1259556512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f2970070ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f29700c48a0 tx=0x7f296002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a7a2c4eaf9116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f29748fd640 1 -- 172.21.15.129:0/1259556512 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960037070 con 0x7f29700c9630 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f29748fd640 1 -- 172.21.15.129:0/1259556512 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296002add0 con 0x7f29700c9630 2024-04-25T01:01:44.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f29748fd640 1 -- 172.21.15.129:0/1259556512 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960032660 con 0x7f29700c9630 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 -- 172.21.15.129:0/1259556512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 msgr2=0x7f2970070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 --2- 172.21.15.129:0/1259556512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f2970070ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f29700c48a0 tx=0x7f296002a080 comp rx=0 tx=0).stop 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 -- 172.21.15.129:0/1259556512 shutdown_connections 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 --2- 172.21.15.129:0/1259556512 >> v2:172.21.15.171:3300/0 conn(0x7f29700c8870 0x7f2970156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 --2- 172.21.15.129:0/1259556512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f2970070ea0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 -- 172.21.15.129:0/1259556512 >> 172.21.15.129:0/1259556512 conn(0x7f2970066ea0 msgr2=0x7f29700672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 -- 172.21.15.129:0/1259556512 shutdown_connections 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.733+0000 7f2977b63640 1 -- 172.21.15.129:0/1259556512 wait complete. 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.734+0000 7f2977b63640 1 Processor -- start 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.734+0000 7f2977b63640 1 -- start start 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2977b63640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29700c75e0 0x7f29700c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2977b63640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f29700c8870 0x7f29701ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2977b63640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f29701de010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2977b63640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f297006fa50 con 0x7f29700c9630 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2977b63640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f297006f8d0 con 0x7f29700c75e0 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2977b63640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f29701eceb0 con 0x7f29700c8870 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f29701de010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f29701de010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34564/0 (socket says 172.21.15.129:34564) 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 -- 172.21.15.129:0/2828250820 learned_addr learned my addr 172.21.15.129:0/2828250820 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f29758ff640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29700c75e0 0x7f29700c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f29750fe640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f29700c8870 0x7f29701ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 -- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f29700c8870 msgr2=0x7f29701ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f29700c8870 0x7f29701ddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 -- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29700c75e0 msgr2=0x7f29700c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29700c75e0 0x7f29700c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 -- 172.21.15.129:0/2828250820 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29701510e0 con 0x7f29700c9630 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f29750fe640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f29700c8870 0x7f29701ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.735+0000 7f2976100640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f29701de010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f296800cd80 tx=0x7f296800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.736+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296800f680 con 0x7f29700c9630 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.736+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296800f840 con 0x7f29700c9630 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.736+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2970150e00 con 0x7f29700c9630 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.736+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296800fc30 con 0x7f29700c9630 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.736+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2970154b10 con 0x7f29700c9630 2024-04-25T01:01:44.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.737+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2970075550 con 0x7f29700c9630 2024-04-25T01:01:44.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.737+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29680213f0 con 0x7f29700c9630 2024-04-25T01:01:44.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.738+0000 7f29667fc640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f295406be60 0x7f295406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:44.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.738+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f296808bde0 con 0x7f29700c9630 2024-04-25T01:01:44.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.738+0000 7f29758ff640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f295406be60 0x7f295406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:44.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.738+0000 7f29758ff640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f295406be60 0x7f295406e320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2960034000 tx=0x7f296002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:44.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.742+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2968014030 con 0x7f29700c9630 2024-04-25T01:01:44.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.926+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f297006fbd0 con 0x7f29700c9630 2024-04-25T01:01:44.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.926+0000 7f29667fc640 1 -- 172.21.15.129:0/2828250820 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f296805a4e0 con 0x7f29700c9630 2024-04-25T01:01:44.927 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T01:01:44.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.928+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f295406be60 msgr2=0x7f295406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.928+0000 7f2977b63640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f295406be60 0x7f295406e320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2960034000 tx=0x7f296002e000 comp rx=0 tx=0).stop 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.928+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 msgr2=0x7f29701de010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.928+0000 7f2977b63640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f29701de010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f296800cd80 tx=0x7f296800a650 comp rx=0 tx=0).stop 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 shutdown_connections 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f295406be60 0x7f295406e320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f29700c8870 0x7f29701ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29700c9630 0x7f29701de010 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 --2- 172.21.15.129:0/2828250820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f29700c75e0 0x7f29700c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 >> 172.21.15.129:0/2828250820 conn(0x7f2970066ea0 msgr2=0x7f2970067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 shutdown_connections 2024-04-25T01:01:44.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:44.929+0000 7f2977b63640 1 -- 172.21.15.129:0/2828250820 wait complete. 2024-04-25T01:01:44.943 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:01:44.943 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:01:44.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:45.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.094+0000 7ff81656a640 1 Processor -- start 2024-04-25T01:01:45.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.094+0000 7ff81656a640 1 -- start start 2024-04-25T01:01:45.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff81656a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff81006d270 0x7ff8100abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff81656a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8101530e0 con 0x7ff8100c6190 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff81656a640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8100c2f60 con 0x7ff810153e40 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff81656a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7ff810153360 con 0x7ff81006d270 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff80ffff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff8100c6190 0x7ff8100c2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60250/0 (socket says 172.21.15.129:60250) 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff80ffff640 1 -- 172.21.15.129:0/2272382439 learned_addr learned my addr 172.21.15.129:0/2272382439 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff80f7fe640 1 --2- 172.21.15.129:0/2272382439 >> v2:172.21.15.171:3300/0 conn(0x7ff81006d270 0x7ff8100abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923945675 0 0) 0x7ff800001110 con 0x7ff8100c6190 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.095+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7f8003520 con 0x7ff8100c6190 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 281544630 0 0) 0x7ff7f8003520 con 0x7ff8100c6190 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 >> v2:172.21.15.171:3300/0 conn(0x7ff81006d270 msgr2=0x7ff8100abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 --2- 172.21.15.129:0/2272382439 >> v2:172.21.15.171:3300/0 conn(0x7ff81006d270 0x7ff8100abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 >> v1:172.21.15.171:6789/0 conn(0x7ff810153e40 legacy=0x7ff8101566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff810153640 con 0x7ff8100c6190 2024-04-25T01:01:45.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7ff800001840 con 0x7ff8100c6190 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 >> v1:172.21.15.129:6789/0 conn(0x7ff8100c6190 legacy=0x7ff8100c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7f80039b0 0x7ff7f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff7f8004800 0x7ff7f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff7f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff810153360 con 0x7ff7f80051c0 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f800c060 con 0x7ff7f80039b0 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f800c260 con 0x7ff7f8004800 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80ffff640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7f80039b0 0x7ff7f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff814b07640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff7f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80f7fe640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff7f8004800 0x7ff7f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff814b07640 1 -- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff7f8004800 msgr2=0x7ff7f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff814b07640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff7f8004800 0x7ff7f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff814b07640 1 -- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7f80039b0 msgr2=0x7ff7f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff814b07640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7f80039b0 0x7ff7f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff814b07640 1 -- 172.21.15.129:0/2272382439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f8003520 con 0x7ff7f80051c0 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.096+0000 7ff80ffff640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7f80039b0 0x7ff7f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff814b07640 1 -- 172.21.15.129:0/2272382439 reap_dead start 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff814b07640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff7f800b9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff808001e20 tx=0x7ff7f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac43108f1cdeb439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80000c0e0 con 0x7ff7f80051c0 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80802ae40 con 0x7ff7f80051c0 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff80effd640 1 -- 172.21.15.129:0/2272382439 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808001a70 con 0x7ff7f80051c0 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff81656a640 1 -- 172.21.15.129:0/2272382439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 msgr2=0x7ff7f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff81656a640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff7f800b9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff808001e20 tx=0x7ff7f8005830 comp rx=0 tx=0).stop 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff81656a640 1 -- 172.21.15.129:0/2272382439 shutdown_connections 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff81656a640 1 --2- 172.21.15.129:0/2272382439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff7f800b9c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.097+0000 7ff81656a640 1 -- 172.21.15.129:0/2272382439 >> 172.21.15.129:0/2272382439 conn(0x7ff8100a86b0 msgr2=0x7ff8100a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.098+0000 7ff81656a640 1 -- 172.21.15.129:0/2272382439 shutdown_connections 2024-04-25T01:01:45.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.098+0000 7ff81656a640 1 -- 172.21.15.129:0/2272382439 wait complete. 2024-04-25T01:01:45.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 Processor -- start 2024-04-25T01:01:45.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 -- start start 2024-04-25T01:01:45.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff8100bc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8101e9070 0x7ff81014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810072750 0x7ff810072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8100c26e0 con 0x7ff7f80051c0 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8100c2560 con 0x7ff810072750 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff81656a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff8100c2860 con 0x7ff8101e9070 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.099+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff8100bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff8100bc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34590/0 (socket says 172.21.15.129:34590) 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 -- 172.21.15.129:0/890942048 learned_addr learned my addr 172.21.15.129:0/890942048 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff814b07640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810072750 0x7ff810072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 -- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8101e9070 msgr2=0x7ff81014d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80f7fe640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8101e9070 0x7ff81014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8101e9070 0x7ff81014d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 -- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810072750 msgr2=0x7ff810072bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810072750 0x7ff810072bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 -- 172.21.15.129:0/890942048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8100bbcd0 con 0x7ff7f80051c0 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff814b07640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810072750 0x7ff810072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80ffff640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff8100bc310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8000072f0 tx=0x7ff8000077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80001a070 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80000f8e0 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8100bbfb0 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80000fc90 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.100+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff810157230 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.101+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff810075550 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.101+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff80001d070 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.102+0000 7ff80cff9640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff7fc06be30 0x7ff7fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.102+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff80008ab50 con 0x7ff7f80051c0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.102+0000 7ff80f7fe640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff7fc06be30 0x7ff7fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.103+0000 7ff80f7fe640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff7fc06be30 0x7ff7fc06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8100a81f0 tx=0x7ff804008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.106+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8101530e0 con 0x7ff7f80051c0 2024-04-25T01:01:45.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.282+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8100c0f10 con 0x7ff7f80051c0 2024-04-25T01:01:45.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.283+0000 7ff80cff9640 1 -- 172.21.15.129:0/890942048 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7ff800059250 con 0x7ff7f80051c0 2024-04-25T01:01:45.284 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:45.284 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:45.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff7fc06be30 msgr2=0x7ff7fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff7fc06be30 0x7ff7fc06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8100a81f0 tx=0x7ff804008040 comp rx=0 tx=0).stop 2024-04-25T01:01:45.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 msgr2=0x7ff8100bc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff8100bc310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8000072f0 tx=0x7ff8000077b0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 shutdown_connections 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff7fc06be30 0x7ff7fc06e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8101e9070 0x7ff81014d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff810072750 0x7ff810072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 --2- 172.21.15.129:0/890942048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7f80051c0 0x7ff8100bc310 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 >> 172.21.15.129:0/890942048 conn(0x7ff8100a86b0 msgr2=0x7ff8100c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 shutdown_connections 2024-04-25T01:01:45.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.285+0000 7ff81656a640 1 -- 172.21.15.129:0/890942048 wait complete. 2024-04-25T01:01:45.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:01:45.298 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:01:45.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.397+0000 7f15efc93640 1 Processor -- start 2024-04-25T01:01:45.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.397+0000 7f15efc93640 1 -- start start 2024-04-25T01:01:45.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15efc93640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e8159630 0x7f15e80c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15efc93640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e8159ac0 con 0x7f15e80c90a0 2024-04-25T01:01:45.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15efc93640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e80c5c10 con 0x7f15e80c8cc0 2024-04-25T01:01:45.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15efc93640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f15e80ca4b0 con 0x7f15e8159630 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15ed22e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f15e80c90a0 0x7f15e815df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60252/0 (socket says 172.21.15.129:60252) 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15ed22e640 1 -- 172.21.15.129:0/2388201500 learned_addr learned my addr 172.21.15.129:0/2388201500 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.398+0000 7f15ee230640 1 --2- 172.21.15.129:0/2388201500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e8159630 0x7f15e80c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310210146 0 0) 0x7f15e8159ac0 con 0x7f15e80c90a0 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15d0003520 con 0x7f15e80c90a0 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937068871 0 0) 0x7f15e80c5c10 con 0x7f15e80c8cc0 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e8159ac0 con 0x7f15e80c8cc0 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15ee230640 1 -- 172.21.15.129:0/2388201500 >> v1:172.21.15.171:6789/0 conn(0x7f15e80c8cc0 legacy=0x7f15e8156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15ee230640 1 -- 172.21.15.129:0/2388201500 >> v1:172.21.15.129:6789/0 conn(0x7f15e80c90a0 legacy=0x7f15e815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2894706616 0 0) 0x7f15d0003520 con 0x7f15e80c90a0 2024-04-25T01:01:45.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15ee230640 1 -- 172.21.15.129:0/2388201500 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e80c5650 con 0x7f15e8159630 2024-04-25T01:01:45.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.399+0000 7f15ee230640 1 --2- 172.21.15.129:0/2388201500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e8159630 0x7f15e80c9f70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15e00097d0 tx=0x7f15e002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8fea5ad28672c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0037070 con 0x7f15e8159630 2024-04-25T01:01:45.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e002ab90 con 0x7f15e8159630 2024-04-25T01:01:45.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15eca2d640 1 -- 172.21.15.129:0/2388201500 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc001670 con 0x7f15e8159630 2024-04-25T01:01:45.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 -- 172.21.15.129:0/2388201500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e8159630 msgr2=0x7f15e80c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 --2- 172.21.15.129:0/2388201500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e8159630 0x7f15e80c9f70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15e00097d0 tx=0x7f15e002a100 comp rx=0 tx=0).stop 2024-04-25T01:01:45.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 -- 172.21.15.129:0/2388201500 shutdown_connections 2024-04-25T01:01:45.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 --2- 172.21.15.129:0/2388201500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e8159630 0x7f15e80c9f70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 -- 172.21.15.129:0/2388201500 >> 172.21.15.129:0/2388201500 conn(0x7f15e806c400 msgr2=0x7f15e806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 -- 172.21.15.129:0/2388201500 shutdown_connections 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.400+0000 7f15efc93640 1 -- 172.21.15.129:0/2388201500 wait complete. 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.401+0000 7f15efc93640 1 Processor -- start 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.401+0000 7f15efc93640 1 -- start start 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15efc93640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15e80c8cc0 0x7f15e80c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15efc93640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e80c90a0 0x7f15e80bf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15efc93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 0x7f15e81eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15efc93640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8155850 con 0x7f15e8159630 2024-04-25T01:01:45.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15efc93640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15e80c9510 con 0x7f15e80c8cc0 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15efc93640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f15e806d1f0 con 0x7f15e80c90a0 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15ee230640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 0x7f15e81eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15ee230640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 0x7f15e81eb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34600/0 (socket says 172.21.15.129:34600) 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15ee230640 1 -- 172.21.15.129:0/3451748564 learned_addr learned my addr 172.21.15.129:0/3451748564 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15ed22e640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e80c90a0 0x7f15e80bf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15eda2f640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15e80c8cc0 0x7f15e80c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15ee230640 1 -- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e80c90a0 msgr2=0x7f15e80bf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.402+0000 7f15ee230640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e80c90a0 0x7f15e80bf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15ee230640 1 -- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15e80c8cc0 msgr2=0x7f15e80c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15ee230640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15e80c8cc0 0x7f15e80c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15ee230640 1 -- 172.21.15.129:0/3451748564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81eb260 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15ee230640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 0x7f15e81eb740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f15e0035040 tx=0x7f15e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0037050 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e0038040 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e80be5d0 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0006e10 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.403+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e80beb90 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.404+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e81eb430 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.404+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15e0003610 con 0x7f15e8159630 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.405+0000 7f15e67fc640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f15d406be60 0x7f15d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.405+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f15e00ab6c0 con 0x7f15e8159630 2024-04-25T01:01:45.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.405+0000 7f15eda2f640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f15d406be60 0x7f15d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.405+0000 7f15eda2f640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f15d406be60 0x7f15d406e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15d8001560 tx=0x7f15d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.409+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15e0079ee0 con 0x7f15e8159630 2024-04-25T01:01:45.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.587+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15e80c9760 con 0x7f15e8159630 2024-04-25T01:01:45.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.587+0000 7f15e67fc640 1 -- 172.21.15.129:0/3451748564 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f15e00797e0 con 0x7f15e8159630 2024-04-25T01:01:45.588 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:01:45.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f15d406be60 msgr2=0x7f15d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f15d406be60 0x7f15d406e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15d8001560 tx=0x7f15d8009040 comp rx=0 tx=0).stop 2024-04-25T01:01:45.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 msgr2=0x7f15e81eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 0x7f15e81eb740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f15e0035040 tx=0x7f15e00046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 shutdown_connections 2024-04-25T01:01:45.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f15d406be60 0x7f15d406e320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15e80c90a0 0x7f15e80bf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15e8159630 0x7f15e81eb740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 --2- 172.21.15.129:0/3451748564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15e80c8cc0 0x7f15e80c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 >> 172.21.15.129:0/3451748564 conn(0x7f15e806c400 msgr2=0x7f15e80c6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:45.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 shutdown_connections 2024-04-25T01:01:45.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.589+0000 7f15efc93640 1 -- 172.21.15.129:0/3451748564 wait complete. 2024-04-25T01:01:45.601 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:01:45.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.699+0000 7f7e19a9e640 1 Processor -- start 2024-04-25T01:01:45.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.699+0000 7f7e19a9e640 1 -- start start 2024-04-25T01:01:45.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e19a9e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7e140c4f00 0x7f7e1406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e19a9e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e140c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e19a9e640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e14153ea0 con 0x7f7e140aa940 2024-04-25T01:01:45.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e19a9e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f7e140c8170 con 0x7f7e140c4f00 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e19a9e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7e140c83c0 con 0x7f7e140c6e70 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e12ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7e140c4f00 0x7f7e1406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e12ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7e140c4f00 0x7f7e1406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34612/0 (socket says 172.21.15.129:34612) 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e12ffd640 1 -- 172.21.15.129:0/592490582 learned_addr learned my addr 172.21.15.129:0/592490582 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.700+0000 7f7e13fff640 1 --2- 172.21.15.129:0/592490582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e140c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e12ffd640 1 -- 172.21.15.129:0/592490582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 msgr2=0x7f7e140c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e12ffd640 1 --2- 172.21.15.129:0/592490582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e140c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e12ffd640 1 -- 172.21.15.129:0/592490582 >> v1:172.21.15.171:6789/0 conn(0x7f7e140aa940 legacy=0x7f7e14070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e12ffd640 1 -- 172.21.15.129:0/592490582 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e140c7810 con 0x7f7e140c4f00 2024-04-25T01:01:45.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e12ffd640 1 --2- 172.21.15.129:0/592490582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e1406e8c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009870 tx=0x7f7e0802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b91658c1f00071c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e127fc640 1 -- 172.21.15.129:0/592490582 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08037070 con 0x7f7e140c4f00 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e127fc640 1 -- 172.21.15.129:0/592490582 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08032440 con 0x7f7e140c4f00 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.701+0000 7f7e127fc640 1 -- 172.21.15.129:0/592490582 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08032810 con 0x7f7e140c4f00 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 -- 172.21.15.129:0/592490582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 msgr2=0x7f7e1406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/592490582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e1406e8c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009870 tx=0x7f7e0802a130 comp rx=0 tx=0).stop 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 -- 172.21.15.129:0/592490582 shutdown_connections 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/592490582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e140c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/592490582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e1406e8c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 -- 172.21.15.129:0/592490582 >> 172.21.15.129:0/592490582 conn(0x7f7e140c5490 msgr2=0x7f7e140c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 -- 172.21.15.129:0/592490582 shutdown_connections 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.702+0000 7f7e19a9e640 1 -- 172.21.15.129:0/592490582 wait complete. 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.703+0000 7f7e19a9e640 1 Processor -- start 2024-04-25T01:01:45.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.703+0000 7f7e19a9e640 1 -- start start 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e19a9e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e140aa940 0x7f7e141dd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e19a9e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e140ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e19a9e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e141e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e19a9e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140711b0 con 0x7f7e140c4f00 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e19a9e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14153b70 con 0x7f7e140aa940 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e19a9e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7e14071330 con 0x7f7e140c6e70 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e140ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e140ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34628/0 (socket says 172.21.15.129:34628) 2024-04-25T01:01:45.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 -- 172.21.15.129:0/3336286656 learned_addr learned my addr 172.21.15.129:0/3336286656 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e137fe640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e140aa940 0x7f7e141dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e13fff640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e141e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 -- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 msgr2=0x7f7e141e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e141e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 -- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e140aa940 msgr2=0x7f7e141dd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e140aa940 0x7f7e141dd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 -- 172.21.15.129:0/3336286656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e141eceb0 con 0x7f7e140c4f00 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.704+0000 7f7e12ffd640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e140ad770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000c00 tx=0x7f7e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.705+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08037040 con 0x7f7e140c4f00 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.705+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08042070 con 0x7f7e140c4f00 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.705+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08030880 con 0x7f7e140c4f00 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.705+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e14157090 con 0x7f7e140c4f00 2024-04-25T01:01:45.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.705+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e140adcb0 con 0x7f7e140c4f00 2024-04-25T01:01:45.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.706+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e141575b0 con 0x7f7e140c4f00 2024-04-25T01:01:45.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.706+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e08030a20 con 0x7f7e140c4f00 2024-04-25T01:01:45.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.706+0000 7f7dfbfff640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e0006be30 0x7f7e0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:45.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.706+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e080ab860 con 0x7f7e140c4f00 2024-04-25T01:01:45.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.707+0000 7f7e137fe640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e0006be30 0x7f7e0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:45.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.707+0000 7f7e137fe640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e0006be30 0x7f7e0006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e04000f60 tx=0x7f7e04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:45.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.711+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e141575b0 con 0x7f7e140c4f00 2024-04-25T01:01:45.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.883+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e141dd130 con 0x7f7e140c4f00 2024-04-25T01:01:45.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.883+0000 7f7dfbfff640 1 -- 172.21.15.129:0/3336286656 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f7e08079f60 con 0x7f7e140c4f00 2024-04-25T01:01:45.884 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e0006be30 msgr2=0x7f7e0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e0006be30 0x7f7e0006e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e04000f60 tx=0x7f7e04009040 comp rx=0 tx=0).stop 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 msgr2=0x7f7e140ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e140ad770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000c00 tx=0x7f7e080046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 shutdown_connections 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e0006be30 0x7f7e0006e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e140c6e70 0x7f7e141e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e140c4f00 0x7f7e140ad770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 --2- 172.21.15.129:0/3336286656 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e140aa940 0x7f7e141dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:45.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 >> 172.21.15.129:0/3336286656 conn(0x7f7e140c5490 msgr2=0x7f7e14150cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:45.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 shutdown_connections 2024-04-25T01:01:45.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.885+0000 7f7e19a9e640 1 -- 172.21.15.129:0/3336286656 wait complete. 2024-04-25T01:01:45.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:01:45.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.996+0000 7fa49d8a5640 1 Processor -- start 2024-04-25T01:01:45.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.996+0000 7fa49d8a5640 1 -- start start 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa49d8a5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498154320 0x7fa4980c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa49d8a5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa4980c4730 0x7fa4980aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa49d8a5640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4980c5f40 con 0x7fa498070fd0 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa49d8a5640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fa498154700 con 0x7fa4980c4730 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa49d8a5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa4980c7320 con 0x7fa498154320 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa4967fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa4980c4730 0x7fa4980aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa4967fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa4980c4730 0x7fa4980aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34632/0 (socket says 172.21.15.129:34632) 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa4967fc640 1 -- 172.21.15.129:0/1941030142 learned_addr learned my addr 172.21.15.129:0/1941030142 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.997+0000 7fa496ffd640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498154320 0x7fa4980c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa4967fc640 1 -- 172.21.15.129:0/1941030142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498154320 msgr2=0x7fa4980c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa4967fc640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498154320 0x7fa4980c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa4967fc640 1 -- 172.21.15.129:0/1941030142 >> v1:172.21.15.171:6789/0 conn(0x7fa498070fd0 legacy=0x7fa4980c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa4967fc640 1 -- 172.21.15.129:0/1941030142 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49806fde0 con 0x7fa4980c4730 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa496ffd640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498154320 0x7fa4980c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa4967fc640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4980c4730 0x7fa4980aa6d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009870 tx=0x7fa48c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42670b846caf2add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa495ffb640 1 -- 172.21.15.129:0/1941030142 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c037070 con 0x7fa4980c4730 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa495ffb640 1 -- 172.21.15.129:0/1941030142 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48c02ae50 con 0x7fa4980c4730 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.998+0000 7fa495ffb640 1 -- 172.21.15.129:0/1941030142 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c032860 con 0x7fa4980c4730 2024-04-25T01:01:46.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 -- 172.21.15.129:0/1941030142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4980c4730 msgr2=0x7fa4980aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4980c4730 0x7fa4980aa6d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009870 tx=0x7fa48c02a300 comp rx=0 tx=0).stop 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 -- 172.21.15.129:0/1941030142 shutdown_connections 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4980c4730 0x7fa4980aa6d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/1941030142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498154320 0x7fa4980c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 -- 172.21.15.129:0/1941030142 >> 172.21.15.129:0/1941030142 conn(0x7fa498066ea0 msgr2=0x7fa4980672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 -- 172.21.15.129:0/1941030142 shutdown_connections 2024-04-25T01:01:46.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:45.999+0000 7fa49d8a5640 1 -- 172.21.15.129:0/1941030142 wait complete. 2024-04-25T01:01:46.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.000+0000 7fa49d8a5640 1 Processor -- start 2024-04-25T01:01:46.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.000+0000 7fa49d8a5640 1 -- start start 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa49d8a5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498070fd0 0x7fa4980acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa49d8a5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4980c4730 0x7fa4980ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa49d8a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 0x7fa4980abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa49d8a5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa49806fa70 con 0x7fa498154320 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa49d8a5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa4980c7850 con 0x7fa4980c4730 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa49d8a5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa4980aac10 con 0x7fa498070fd0 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 0x7fa4980abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 0x7fa4980abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34636/0 (socket says 172.21.15.129:34636) 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 -- 172.21.15.129:0/3225737911 learned_addr learned my addr 172.21.15.129:0/3225737911 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa496ffd640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498070fd0 0x7fa4980acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4967fc640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4980c4730 0x7fa4980ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 -- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498070fd0 msgr2=0x7fa4980acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498070fd0 0x7fa4980acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 -- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4980c4730 msgr2=0x7fa4980ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4980c4730 0x7fa4980ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.001+0000 7fa4977fe640 1 -- 172.21.15.129:0/3225737911 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa498157660 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.002+0000 7fa4977fe640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 0x7fa4980abef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c3a0 tx=0x7fa49000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.002+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490017070 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.002+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4900052d0 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.002+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490003830 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.002+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981568a0 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.002+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa498156e60 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.003+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa498152a70 con 0x7fa498154320 2024-04-25T01:01:46.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.003+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa49001a070 con 0x7fa498154320 2024-04-25T01:01:46.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.004+0000 7fa47ffff640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa48406be30 0x7fa48406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.004+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa49008a4e0 con 0x7fa498154320 2024-04-25T01:01:46.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.004+0000 7fa496ffd640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa48406be30 0x7fa48406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.004+0000 7fa496ffd640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa48406be30 0x7fa48406e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa488007510 tx=0x7fa4880074a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.008+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa498152a70 con 0x7fa498154320 2024-04-25T01:01:46.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.182+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4980ad820 con 0x7fa498154320 2024-04-25T01:01:46.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.183+0000 7fa47ffff640 1 -- 172.21.15.129:0/3225737911 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fa490058c10 con 0x7fa498154320 2024-04-25T01:01:46.183 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa48406be30 msgr2=0x7fa48406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa48406be30 0x7fa48406e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa488007510 tx=0x7fa4880074a0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 msgr2=0x7fa4980abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 0x7fa4980abef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c3a0 tx=0x7fa49000c860 comp rx=0 tx=0).stop 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 shutdown_connections 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa48406be30 0x7fa48406e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa498154320 0x7fa4980abef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4980c4730 0x7fa4980ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 --2- 172.21.15.129:0/3225737911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa498070fd0 0x7fa4980acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 >> 172.21.15.129:0/3225737911 conn(0x7fa498066ea0 msgr2=0x7fa498067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:46.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 shutdown_connections 2024-04-25T01:01:46.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.185+0000 7fa49d8a5640 1 -- 172.21.15.129:0/3225737911 wait complete. 2024-04-25T01:01:46.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:01:46.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.295+0000 7f57d4a12640 1 Processor -- start 2024-04-25T01:01:46.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.295+0000 7f57d4a12640 1 -- start start 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.295+0000 7f57d4a12640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d0153e90 0x7f57d0156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57d4a12640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d00a61e0 con 0x7f57d00a5ce0 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57d4a12640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d01533b0 con 0x7f57d0150750 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57d4a12640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f57d0153690 con 0x7f57d0153e90 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57ced9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f57d00a5ce0 0x7f57d015db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60256/0 (socket says 172.21.15.129:60256) 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57ced9d640 1 -- 172.21.15.129:0/3300808981 learned_addr learned my addr 172.21.15.129:0/3300808981 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:46.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57c7fff640 1 --2- 172.21.15.129:0/3300808981 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d0153e90 0x7f57d0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1277662372 0 0) 0x7f57d00a61e0 con 0x7f57d00a5ce0 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57bc003520 con 0x7f57d00a5ce0 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 610177884 0 0) 0x7f57d01533b0 con 0x7f57d0150750 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57d00a61e0 con 0x7f57d0150750 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57c7fff640 1 -- 172.21.15.129:0/3300808981 >> v1:172.21.15.171:6789/0 conn(0x7f57d0150750 legacy=0x7f57d00abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57c7fff640 1 -- 172.21.15.129:0/3300808981 >> v1:172.21.15.129:6789/0 conn(0x7f57d00a5ce0 legacy=0x7f57d015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3758578514 0 0) 0x7f57bc003520 con 0x7f57d00a5ce0 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.296+0000 7f57c7fff640 1 -- 172.21.15.129:0/3300808981 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d0153970 con 0x7f57d0153e90 2024-04-25T01:01:46.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.297+0000 7f57c7fff640 1 --2- 172.21.15.129:0/3300808981 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d0153e90 0x7f57d0156790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f57b80097d0 tx=0x7f57b802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7b3381f4f89213a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.297+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8037070 con 0x7f57d0153e90 2024-04-25T01:01:46.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.297+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b802ac30 con 0x7f57d0153e90 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.297+0000 7f57cdd9b640 1 -- 172.21.15.129:0/3300808981 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c80019f0 con 0x7f57d0153e90 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 -- 172.21.15.129:0/3300808981 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d0153e90 msgr2=0x7f57d0156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3300808981 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d0153e90 0x7f57d0156790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f57b80097d0 tx=0x7f57b802a1a0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 -- 172.21.15.129:0/3300808981 shutdown_connections 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3300808981 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d0153e90 0x7f57d0156790 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 -- 172.21.15.129:0/3300808981 >> 172.21.15.129:0/3300808981 conn(0x7f57d00a86b0 msgr2=0x7f57d00a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 -- 172.21.15.129:0/3300808981 shutdown_connections 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.298+0000 7f57d4a12640 1 -- 172.21.15.129:0/3300808981 wait complete. 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.299+0000 7f57d4a12640 1 Processor -- start 2024-04-25T01:01:46.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.299+0000 7f57d4a12640 1 -- start start 2024-04-25T01:01:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d00a5ce0 0x7f57d00bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 0x7f57d0072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d0153e90 0x7f57d00728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00ac550 con 0x7f57d0150750 2024-04-25T01:01:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00ac3d0 con 0x7f57d0153e90 2024-04-25T01:01:46.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f57d006c270 con 0x7f57d00a5ce0 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 0x7f57d0072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 0x7f57d0072380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34642/0 (socket says 172.21.15.129:34642) 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 -- 172.21.15.129:0/3324136622 learned_addr learned my addr 172.21.15.129:0/3324136622 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57ced9d640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d0153e90 0x7f57d00728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57ce59c640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d00a5ce0 0x7f57d00bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 -- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d00a5ce0 msgr2=0x7f57d00bffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d00a5ce0 0x7f57d00bffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 -- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d0153e90 msgr2=0x7f57d00728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d0153e90 0x7f57d00728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 -- 172.21.15.129:0/3324136622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d01eb350 con 0x7f57d0150750 2024-04-25T01:01:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c7fff640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 0x7f57d0072380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f57b8035040 tx=0x7f57b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8037050 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b8043070 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d00c4590 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b803ca50 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.300+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d00c04e0 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.301+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d0075550 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.302+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57b803cbf0 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.302+0000 7f57c6ffd640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57b006be30 0x7f57b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.302+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f57b80aba80 con 0x7f57d0150750 2024-04-25T01:01:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.303+0000 7f57ce59c640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57b006be30 0x7f57b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.303+0000 7f57ce59c640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57b006be30 0x7f57b006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57c00011e0 tx=0x7f57c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.307+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57b8079c50 con 0x7f57d0150750 2024-04-25T01:01:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.483+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f57d00c0fe0 con 0x7f57d0150750 2024-04-25T01:01:46.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.483+0000 7f57c6ffd640 1 -- 172.21.15.129:0/3324136622 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f57b807df50 con 0x7f57d0150750 2024-04-25T01:01:46.484 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:01:46.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57b006be30 msgr2=0x7f57b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57b006be30 0x7f57b006e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57c00011e0 tx=0x7f57c0009040 comp rx=0 tx=0).stop 2024-04-25T01:01:46.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 msgr2=0x7f57d0072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 0x7f57d0072380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f57b8035040 tx=0x7f57b80046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 shutdown_connections 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57b006be30 0x7f57b006e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d0153e90 0x7f57d00728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57d0150750 0x7f57d0072380 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 --2- 172.21.15.129:0/3324136622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57d00a5ce0 0x7f57d00bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 >> 172.21.15.129:0/3324136622 conn(0x7f57d00a86b0 msgr2=0x7f57d00abcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 shutdown_connections 2024-04-25T01:01:46.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.485+0000 7f57d4a12640 1 -- 172.21.15.129:0/3324136622 wait complete. 2024-04-25T01:01:46.497 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:01:46.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.595+0000 7fd228b5c640 1 Processor -- start 2024-04-25T01:01:46.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.595+0000 7fd228b5c640 1 -- start start 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd228b5c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd2240c2f70 0x7fd2240bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd228b5c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2240a5ce0 0x7fd2240a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd228b5c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd224155fa0 con 0x7fd224155400 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd228b5c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd22406eb60 con 0x7fd2240c2f70 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd228b5c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd224073740 con 0x7fd2240a5ce0 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd2240c2f70 0x7fd2240bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd2240c2f70 0x7fd2240bffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34652/0 (socket says 172.21.15.129:34652) 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 -- 172.21.15.129:0/184947712 learned_addr learned my addr 172.21.15.129:0/184947712 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd21bfff640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2240a5ce0 0x7fd2240a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 -- 172.21.15.129:0/184947712 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2240a5ce0 msgr2=0x7fd2240a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:46.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2240a5ce0 0x7fd2240a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 -- 172.21.15.129:0/184947712 >> v1:172.21.15.129:6789/0 conn(0x7fd224155400 legacy=0x7fd224155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd22259c640 1 -- 172.21.15.129:0/184947712 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2240c0800 con 0x7fd2240c2f70 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd221d9b640 1 -- 172.21.15.129:0/184947712 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890454948 0 0) 0x7fd224155fa0 con 0x7fd224155400 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.596+0000 7fd21bfff640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2240a5ce0 0x7fd2240a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.597+0000 7fd22259c640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd2240bffb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd2140098a0 tx=0x7fd21402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05161cf9c963687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.597+0000 7fd221d9b640 1 -- 172.21.15.129:0/184947712 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214037070 con 0x7fd2240c2f70 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.597+0000 7fd221d9b640 1 -- 172.21.15.129:0/184947712 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd214032440 con 0x7fd2240c2f70 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.597+0000 7fd221d9b640 1 -- 172.21.15.129:0/184947712 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214032810 con 0x7fd2240c2f70 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 -- 172.21.15.129:0/184947712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 msgr2=0x7fd2240bffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd2240bffb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd2140098a0 tx=0x7fd21402a290 comp rx=0 tx=0).stop 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 -- 172.21.15.129:0/184947712 shutdown_connections 2024-04-25T01:01:46.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2240a5ce0 0x7fd2240a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 --2- 172.21.15.129:0/184947712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd2240bffb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 -- 172.21.15.129:0/184947712 >> 172.21.15.129:0/184947712 conn(0x7fd2240c3680 msgr2=0x7fd2240c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:46.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 -- 172.21.15.129:0/184947712 shutdown_connections 2024-04-25T01:01:46.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.598+0000 7fd228b5c640 1 -- 172.21.15.129:0/184947712 wait complete. 2024-04-25T01:01:46.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.599+0000 7fd228b5c640 1 Processor -- start 2024-04-25T01:01:46.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.599+0000 7fd228b5c640 1 -- start start 2024-04-25T01:01:46.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2240a5ce0 0x7fd224151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd224152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd224155400 0x7fd2241ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd22406e080 con 0x7fd2240c2f70 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd22406e6d0 con 0x7fd2240a5ce0 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd22406e360 con 0x7fd224155400 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd224152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd224152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34662/0 (socket says 172.21.15.129:34662) 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 -- 172.21.15.129:0/3850516276 learned_addr learned my addr 172.21.15.129:0/3850516276 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd22259c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2240a5ce0 0x7fd224151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd222d9d640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd224155400 0x7fd2241ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 -- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd224155400 msgr2=0x7fd2241ee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd224155400 0x7fd2241ee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 -- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2240a5ce0 msgr2=0x7fd224151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2240a5ce0 0x7fd224151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 -- 172.21.15.129:0/3850516276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22409ac70 con 0x7fd2240c2f70 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21bfff640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd224152430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c820 tx=0x7fd20c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c015070 con 0x7fd2240c2f70 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20c007a30 con 0x7fd2240c2f70 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22409af50 con 0x7fd2240c2f70 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.600+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c007de0 con 0x7fd2240c2f70 2024-04-25T01:01:46.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.601+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd224157480 con 0x7fd2240c2f70 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.601+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd224075550 con 0x7fd2240c2f70 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.602+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd20c018070 con 0x7fd2240c2f70 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.602+0000 7fd21affd640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd20406be30 0x7fd20406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.602+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd20c08b010 con 0x7fd2240c2f70 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.602+0000 7fd22259c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd20406be30 0x7fd20406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.603+0000 7fd22259c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd20406be30 0x7fd20406e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd214034000 tx=0x7fd21402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:46.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.607+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd20c059030 con 0x7fd2240c2f70 2024-04-25T01:01:46.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.791+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd2240c5b30 con 0x7fd2240c2f70 2024-04-25T01:01:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.791+0000 7fd21affd640 1 -- 172.21.15.129:0/3850516276 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fd20c05d330 con 0x7fd2240c2f70 2024-04-25T01:01:46.792 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd20406be30 msgr2=0x7fd20406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd20406be30 0x7fd20406e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd214034000 tx=0x7fd21402e000 comp rx=0 tx=0).stop 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 msgr2=0x7fd224152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd224152430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c820 tx=0x7fd20c00cce0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 shutdown_connections 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd20406be30 0x7fd20406e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd224155400 0x7fd2241ee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2240c2f70 0x7fd224152430 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 --2- 172.21.15.129:0/3850516276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2240a5ce0 0x7fd224151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:46.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 >> 172.21.15.129:0/3850516276 conn(0x7fd2240c3680 msgr2=0x7fd224153270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:46.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 shutdown_connections 2024-04-25T01:01:46.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:46.793+0000 7fd228b5c640 1 -- 172.21.15.129:0/3850516276 wait complete. 2024-04-25T01:01:46.804 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi129.front.sepia.ceph.com... 2024-04-25T01:01:46.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:46.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:01:46.826 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:ceph-fuse[41355]: fuse finished with error 0 and tester_r 0 2024-04-25T01:01:46.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:01:52.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:01:52.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:01:52.930 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T01:01:53.033 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:53.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.141+0000 7f97cd563640 1 Processor -- start 2024-04-25T01:01:53.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.141+0000 7f97cd563640 1 -- start start 2024-04-25T01:01:53.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97cd563640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f97c8152fa0 0x7f97c80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97cd563640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c80c9860 con 0x7f97c81561e0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97cd563640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c80c5a70 con 0x7f97c80c5520 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97cd563640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f97c80c5d50 con 0x7f97c8152fa0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c6ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f97c8152fa0 0x7f97c80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c77fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f97c81561e0 0x7f97c80c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:57346/0 (socket says 172.21.15.129:57346) 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c6ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f97c8152fa0 0x7f97c80ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45382/0 (socket says 172.21.15.129:45382) 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c77fe640 1 -- 172.21.15.129:0/1720240300 learned_addr learned my addr 172.21.15.129:0/1720240300 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c6ffd640 1 -- 172.21.15.129:0/1720240300 >> v1:172.21.15.171:6789/0 conn(0x7f97c80c5520 legacy=0x7f97c80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c6ffd640 1 -- 172.21.15.129:0/1720240300 >> v1:172.21.15.129:6789/0 conn(0x7f97c81561e0 legacy=0x7f97c80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c67fc640 1 -- 172.21.15.129:0/1720240300 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794935419 0 0) 0x7f97c0001490 con 0x7f97c81561e0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.142+0000 7f97c6ffd640 1 -- 172.21.15.129:0/1720240300 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c80c7650 con 0x7f97c8152fa0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.143+0000 7f97c6ffd640 1 --2- 172.21.15.129:0/1720240300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c8152fa0 0x7f97c80ca2e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097a0 tx=0x7f97b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d8dc45eb67d338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.143+0000 7f97c67fc640 1 -- 172.21.15.129:0/1720240300 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8037070 con 0x7f97c8152fa0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.143+0000 7f97c67fc640 1 -- 172.21.15.129:0/1720240300 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b8032440 con 0x7f97c8152fa0 2024-04-25T01:01:53.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.143+0000 7f97c67fc640 1 -- 172.21.15.129:0/1720240300 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b802ae60 con 0x7f97c8152fa0 2024-04-25T01:01:53.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 -- 172.21.15.129:0/1720240300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c8152fa0 msgr2=0x7f97c80ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 --2- 172.21.15.129:0/1720240300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c8152fa0 0x7f97c80ca2e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097a0 tx=0x7f97b802a110 comp rx=0 tx=0).stop 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 -- 172.21.15.129:0/1720240300 shutdown_connections 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 --2- 172.21.15.129:0/1720240300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c8152fa0 0x7f97c80ca2e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 -- 172.21.15.129:0/1720240300 >> 172.21.15.129:0/1720240300 conn(0x7f97c806c400 msgr2=0x7f97c806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 -- 172.21.15.129:0/1720240300 shutdown_connections 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.144+0000 7f97cd563640 1 -- 172.21.15.129:0/1720240300 wait complete. 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 Processor -- start 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 -- start start 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 0x7f97c80c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97c8152fa0 0x7f97c81ef820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97c81561e0 0x7f97c8151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80c93d0 con 0x7f97c80c5520 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80c6030 con 0x7f97c81561e0 2024-04-25T01:01:53.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97cd563640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f97c80c9550 con 0x7f97c8152fa0 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.145+0000 7f97c6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 0x7f97c80c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 0x7f97c80c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45396/0 (socket says 172.21.15.129:45396) 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 -- 172.21.15.129:0/3485730988 learned_addr learned my addr 172.21.15.129:0/3485730988 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c77fe640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97c81561e0 0x7f97c8151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97bffff640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97c8152fa0 0x7f97c81ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 -- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97c8152fa0 msgr2=0x7f97c81ef820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97c8152fa0 0x7f97c81ef820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 -- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97c81561e0 msgr2=0x7f97c8151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97c81561e0 0x7f97c8151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 -- 172.21.15.129:0/3485730988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c8152480 con 0x7f97c80c5520 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97bffff640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97c8152fa0 0x7f97c81ef820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c77fe640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97c81561e0 0x7f97c8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97c6ffd640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 0x7f97c80c20c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097a0 tx=0x7f97b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8037050 con 0x7f97c80c5520 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b8038040 con 0x7f97c80c5520 2024-04-25T01:01:53.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81eb8a0 con 0x7f97c80c5520 2024-04-25T01:01:53.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004470 con 0x7f97c80c5520 2024-04-25T01:01:53.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.146+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c8158b90 con 0x7f97c80c5520 2024-04-25T01:01:53.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.147+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c8075550 con 0x7f97c80c5520 2024-04-25T01:01:53.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.147+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97b8030900 con 0x7f97c80c5520 2024-04-25T01:01:53.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.148+0000 7f97bf7fe640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f97a806be30 0x7f97a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.148+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f97b80abeb0 con 0x7f97c80c5520 2024-04-25T01:01:53.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.148+0000 7f97bffff640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f97a806be30 0x7f97a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.149+0000 7f97bffff640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f97a806be30 0x7f97a806e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97b0007fd0 tx=0x7f97b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.152+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f97b807a6d0 con 0x7f97c80c5520 2024-04-25T01:01:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.325+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97c8156880 con 0x7f97c80c5520 2024-04-25T01:01:53.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.326+0000 7f97bf7fe640 1 -- 172.21.15.129:0/3485730988 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f97b8079fd0 con 0x7f97c80c5520 2024-04-25T01:01:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:53.332 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:53.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f97a806be30 msgr2=0x7f97a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f97a806be30 0x7f97a806e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97b0007fd0 tx=0x7f97b0009040 comp rx=0 tx=0).stop 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 msgr2=0x7f97c80c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 0x7f97c80c20c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097a0 tx=0x7f97b80046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 shutdown_connections 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f97a806be30 0x7f97a806e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f97c8152fa0 0x7f97c81ef820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97c81561e0 0x7f97c8151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 --2- 172.21.15.129:0/3485730988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97c80c5520 0x7f97c80c20c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 >> 172.21.15.129:0/3485730988 conn(0x7f97c806c400 msgr2=0x7f97c806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 shutdown_connections 2024-04-25T01:01:53.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.328+0000 7f97cd563640 1 -- 172.21.15.129:0/3485730988 wait complete. 2024-04-25T01:01:53.341 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:53.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.437+0000 7f05eb1d5640 1 Processor -- start 2024-04-25T01:01:53.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05eb1d5640 1 -- start start 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05eb1d5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e4152fa0 0x7f05e40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05eb1d5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f05e41561e0 0x7f05e40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05eb1d5640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e40c9860 con 0x7f05e40c5520 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05eb1d5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f05e4156690 con 0x7f05e41561e0 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05eb1d5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f05e40c5a70 con 0x7f05e4152fa0 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05e3fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f05e40c5520 0x7f05e40c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:57354/0 (socket says 172.21.15.129:57354) 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05e3fff640 1 -- 172.21.15.129:0/3646219289 learned_addr learned my addr 172.21.15.129:0/3646219289 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:53.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05e8f71640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e4152fa0 0x7f05e40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.438+0000 7f05e9772640 1 --2- 172.21.15.129:0/3646219289 >> v2:172.21.15.171:3300/0 conn(0x7f05e41561e0 0x7f05e40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e37fe640 1 -- 172.21.15.129:0/3646219289 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638177526 0 0) 0x7f05d0001110 con 0x7f05e40c5520 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e37fe640 1 -- 172.21.15.129:0/3646219289 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05d4003520 con 0x7f05e40c5520 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e9772640 1 -- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e4152fa0 msgr2=0x7f05e40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e9772640 1 --2- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e4152fa0 0x7f05e40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e9772640 1 -- 172.21.15.129:0/3646219289 >> v1:172.21.15.129:6789/0 conn(0x7f05e40c5520 legacy=0x7f05e40c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e9772640 1 -- 172.21.15.129:0/3646219289 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40c5d50 con 0x7f05e41561e0 2024-04-25T01:01:53.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.439+0000 7f05e8f71640 1 --2- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e4152fa0 0x7f05e40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:53.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.440+0000 7f05e9772640 1 --2- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e41561e0 0x7f05e40c8cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009d90 tx=0x7f05dc02a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d27aaae99cff7ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.442 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.440+0000 7f05e37fe640 1 -- 172.21.15.129:0/3646219289 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007740 con 0x7f05e41561e0 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.440+0000 7f05e37fe640 1 -- 172.21.15.129:0/3646219289 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc0078e0 con 0x7f05e41561e0 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.440+0000 7f05e37fe640 1 -- 172.21.15.129:0/3646219289 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007cb0 con 0x7f05e41561e0 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 -- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e41561e0 msgr2=0x7f05e40c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e41561e0 0x7f05e40c8cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009d90 tx=0x7f05dc02a4e0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 -- 172.21.15.129:0/3646219289 shutdown_connections 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e41561e0 0x7f05e40c8cc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/3646219289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e4152fa0 0x7f05e40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 -- 172.21.15.129:0/3646219289 >> 172.21.15.129:0/3646219289 conn(0x7f05e406c400 msgr2=0x7f05e406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 -- 172.21.15.129:0/3646219289 shutdown_connections 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.441+0000 7f05eb1d5640 1 -- 172.21.15.129:0/3646219289 wait complete. 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.442+0000 7f05eb1d5640 1 Processor -- start 2024-04-25T01:01:53.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.442+0000 7f05eb1d5640 1 -- start start 2024-04-25T01:01:53.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05eb1d5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e40c5520 0x7f05e40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05eb1d5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e4152fa0 0x7f05e414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05eb1d5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 0x7f05e414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05eb1d5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05e40c9200 con 0x7f05e41561e0 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05eb1d5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f05e40c6030 con 0x7f05e4152fa0 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05eb1d5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f05e40c9380 con 0x7f05e40c5520 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 0x7f05e414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 0x7f05e414beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45404/0 (socket says 172.21.15.129:45404) 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 -- 172.21.15.129:0/4019956846 learned_addr learned my addr 172.21.15.129:0/4019956846 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e3fff640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e4152fa0 0x7f05e414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e8f71640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e40c5520 0x7f05e40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 -- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e40c5520 msgr2=0x7f05e40c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e40c5520 0x7f05e40c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 -- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e4152fa0 msgr2=0x7f05e414b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e4152fa0 0x7f05e414b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.443+0000 7f05e9772640 1 -- 172.21.15.129:0/4019956846 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e41ec250 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.444+0000 7f05e9772640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 0x7f05e414beb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009eb0 tx=0x7f05dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.444+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc040070 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.444+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc031bc0 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.444+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc009450 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.444+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e40c1da0 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.444+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e40c2360 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.445+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e4075550 con 0x7f05e41561e0 2024-04-25T01:01:53.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.445+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05dc039040 con 0x7f05e41561e0 2024-04-25T01:01:53.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.446+0000 7f05e17fa640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f05c806be30 0x7f05c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.446+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f05dc0ac770 con 0x7f05e41561e0 2024-04-25T01:01:53.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.446+0000 7f05e8f71640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f05c806be30 0x7f05c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.446+0000 7f05e8f71640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f05c806be30 0x7f05c806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009560 tx=0x7f05d8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.450+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05dc07a940 con 0x7f05e41561e0 2024-04-25T01:01:53.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.628+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05e40ca0c0 con 0x7f05e41561e0 2024-04-25T01:01:53.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.629+0000 7f05e17fa640 1 -- 172.21.15.129:0/4019956846 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f05dc07ec40 con 0x7f05e41561e0 2024-04-25T01:01:53.630 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:53.630 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f05c806be30 msgr2=0x7f05c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f05c806be30 0x7f05c806e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009560 tx=0x7f05d8009190 comp rx=0 tx=0).stop 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 msgr2=0x7f05e414beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 0x7f05e414beb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009eb0 tx=0x7f05dc004a40 comp rx=0 tx=0).stop 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 shutdown_connections 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f05c806be30 0x7f05c806e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41561e0 0x7f05e414beb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05e4152fa0 0x7f05e414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 --2- 172.21.15.129:0/4019956846 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05e40c5520 0x7f05e40c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 >> 172.21.15.129:0/4019956846 conn(0x7f05e406c400 msgr2=0x7f05e40ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 shutdown_connections 2024-04-25T01:01:53.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.631+0000 7f05eb1d5640 1 -- 172.21.15.129:0/4019956846 wait complete. 2024-04-25T01:01:53.643 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T01:01:53.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 Processor -- start 2024-04-25T01:01:53.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 -- start start 2024-04-25T01:01:53.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 0x7fa9400c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa9400c9f70 0x7fa94015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400c5c10 con 0x7fa940159630 2024-04-25T01:01:53.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fa9400c5650 con 0x7fa9400c9f70 2024-04-25T01:01:53.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.742+0000 7fa946363640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400c5930 con 0x7fa9400c8cc0 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 0x7fa9400c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa9400c9f70 0x7fa94015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa9400c9f70 0x7fa94015ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34024/0 (socket says 172.21.15.129:34024) 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 0x7fa9400c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:51686/0 (socket says 172.21.15.129:51686) 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 -- 172.21.15.129:0/395566587 learned_addr learned my addr 172.21.15.129:0/395566587 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa93f7fe640 1 --1- 172.21.15.129:0/395566587 >> v1:172.21.15.171:6789/0 conn(0x7fa940159630 0x7fa940156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:59038/0 (socket says 172.21.15.129:59038) 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 -- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 msgr2=0x7fa9400c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 --2- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 0x7fa9400c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 -- 172.21.15.129:0/395566587 >> v1:172.21.15.171:6789/0 conn(0x7fa940159630 legacy=0x7fa940156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa944900640 1 -- 172.21.15.129:0/395566587 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa940155850 con 0x7fa9400c9f70 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.743+0000 7fa93ffff640 1 --2- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 0x7fa9400c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa944900640 1 --2- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9400c9f70 0x7fa94015ff50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa938009e50 tx=0x7fa93802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71e7d2388dd2661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa93effd640 1 -- 172.21.15.129:0/395566587 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938036070 con 0x7fa9400c9f70 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa93effd640 1 -- 172.21.15.129:0/395566587 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938031600 con 0x7fa9400c9f70 2024-04-25T01:01:53.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa93effd640 1 -- 172.21.15.129:0/395566587 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380319d0 con 0x7fa9400c9f70 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa946363640 1 -- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9400c9f70 msgr2=0x7fa94015ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa946363640 1 --2- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9400c9f70 0x7fa94015ff50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa938009e50 tx=0x7fa93802a680 comp rx=0 tx=0).stop 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa946363640 1 -- 172.21.15.129:0/395566587 shutdown_connections 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa946363640 1 --2- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9400c9f70 0x7fa94015ff50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa946363640 1 --2- 172.21.15.129:0/395566587 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c8cc0 0x7fa9400c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.744+0000 7fa946363640 1 -- 172.21.15.129:0/395566587 >> 172.21.15.129:0/395566587 conn(0x7fa94006c400 msgr2=0x7fa94006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.745+0000 7fa946363640 1 -- 172.21.15.129:0/395566587 shutdown_connections 2024-04-25T01:01:53.748 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.745+0000 7fa946363640 1 -- 172.21.15.129:0/395566587 wait complete. 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 Processor -- start 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 -- start start 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 0x7fa9401dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c9f70 0x7fa940154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa940159630 0x7fa940145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa94006d1f0 con 0x7fa9400c8cc0 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400ca400 con 0x7fa940159630 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.746+0000 7fa946363640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400c5f30 con 0x7fa9400c9f70 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 0x7fa9401dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 0x7fa9401dfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45410/0 (socket says 172.21.15.129:45410) 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 -- 172.21.15.129:0/1809419057 learned_addr learned my addr 172.21.15.129:0/1809419057 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa944900640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa940159630 0x7fa940145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93f7fe640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c9f70 0x7fa940154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 -- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c9f70 msgr2=0x7fa940154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c9f70 0x7fa940154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 -- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa940159630 msgr2=0x7fa940145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa940159630 0x7fa940145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 -- 172.21.15.129:0/1809419057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9401ef1b0 con 0x7fa9400c8cc0 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93ffff640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 0x7fa9401dfd20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa93400e960 tx=0x7fa93400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934017070 con 0x7fa9400c8cc0 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93400ca00 con 0x7fa9400c8cc0 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9401ef490 con 0x7fa9400c8cc0 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93400cdd0 con 0x7fa9400c8cc0 2024-04-25T01:01:53.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.747+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9401eb700 con 0x7fa9400c8cc0 2024-04-25T01:01:53.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.748+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9401eb230 con 0x7fa9400c8cc0 2024-04-25T01:01:53.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.749+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa93401a070 con 0x7fa9400c8cc0 2024-04-25T01:01:53.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.749+0000 7fa93cff9640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa92c06be30 0x7fa92c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:53.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.749+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa93408b2d0 con 0x7fa9400c8cc0 2024-04-25T01:01:53.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.749+0000 7fa93f7fe640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa92c06be30 0x7fa92c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:53.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.750+0000 7fa93f7fe640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa92c06be30 0x7fa92c06e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa940158d10 tx=0x7fa930007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:53.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.754+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa934059380 con 0x7fa9400c8cc0 2024-04-25T01:01:53.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:53.965+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa940075550 con 0x7fa9400c8cc0 2024-04-25T01:01:54.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.466+0000 7fa93cff9640 1 -- 172.21.15.129:0/1809419057 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fa93405d680 con 0x7fa9400c8cc0 2024-04-25T01:01:54.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.468+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa92c06be30 msgr2=0x7fa92c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:54.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.468+0000 7fa946363640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa92c06be30 0x7fa92c06e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa940158d10 tx=0x7fa930007e80 comp rx=0 tx=0).stop 2024-04-25T01:01:54.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 msgr2=0x7fa9401dfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:54.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 0x7fa9401dfd20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa93400e960 tx=0x7fa93400ee20 comp rx=0 tx=0).stop 2024-04-25T01:01:54.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 shutdown_connections 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa92c06be30 0x7fa92c06e2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9400c9f70 0x7fa940154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa940159630 0x7fa940145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 --2- 172.21.15.129:0/1809419057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9400c8cc0 0x7fa9401dfd20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 >> 172.21.15.129:0/1809419057 conn(0x7fa94006c400 msgr2=0x7fa9400abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 shutdown_connections 2024-04-25T01:01:54.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.469+0000 7fa946363640 1 -- 172.21.15.129:0/1809419057 wait complete. 2024-04-25T01:01:54.480 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T01:01:54.481 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T01:01:54.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.580+0000 7f1b7a672640 1 Processor -- start 2024-04-25T01:01:54.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.580+0000 7f1b7a672640 1 -- start start 2024-04-25T01:01:54.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b7a672640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1b74152cb0 0x7f1b740c6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b7a672640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1b740c8480 0x7f1b740c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b7a672640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b740c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b7a672640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f1b74071310 con 0x7f1b740c8480 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b7a672640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f1b740c3670 con 0x7f1b74152cb0 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b7a672640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1b74159740 con 0x7f1b740c3930 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1b740c8480 0x7f1b740c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1b740c8480 0x7f1b740c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45416/0 (socket says 172.21.15.129:45416) 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 -- 172.21.15.129:0/47989158 learned_addr learned my addr 172.21.15.129:0/47989158 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b73fff640 1 --2- 172.21.15.129:0/47989158 >> v2:172.21.15.171:3300/0 conn(0x7f1b74152cb0 0x7f1b740c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b78c0f640 1 --2- 172.21.15.129:0/47989158 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b740c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 -- 172.21.15.129:0/47989158 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 msgr2=0x7f1b740c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 --2- 172.21.15.129:0/47989158 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b740c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 -- 172.21.15.129:0/47989158 >> v2:172.21.15.171:3300/0 conn(0x7f1b74152cb0 msgr2=0x7f1b740c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 --2- 172.21.15.129:0/47989158 >> v2:172.21.15.171:3300/0 conn(0x7f1b74152cb0 0x7f1b740c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b737fe640 1 -- 172.21.15.129:0/47989158 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b740c6ed0 con 0x7f1b740c8480 2024-04-25T01:01:54.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.581+0000 7f1b73fff640 1 --2- 172.21.15.129:0/47989158 >> v2:172.21.15.171:3300/0 conn(0x7f1b74152cb0 0x7f1b740c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:54.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b737fe640 1 --2- 172.21.15.129:0/47989158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b740c1dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1b74075290 tx=0x7f1b6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabda7c067adc211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:54.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b72ffd640 1 -- 172.21.15.129:0/47989158 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64037070 con 0x7f1b740c8480 2024-04-25T01:01:54.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b72ffd640 1 -- 172.21.15.129:0/47989158 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64032440 con 0x7f1b740c8480 2024-04-25T01:01:54.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b72ffd640 1 -- 172.21.15.129:0/47989158 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64032810 con 0x7f1b740c8480 2024-04-25T01:01:54.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 -- 172.21.15.129:0/47989158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 msgr2=0x7f1b740c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:54.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 --2- 172.21.15.129:0/47989158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b740c1dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1b74075290 tx=0x7f1b6402a1e0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 -- 172.21.15.129:0/47989158 shutdown_connections 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 --2- 172.21.15.129:0/47989158 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b740c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 --2- 172.21.15.129:0/47989158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b740c1dc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 --2- 172.21.15.129:0/47989158 >> v2:172.21.15.171:3300/0 conn(0x7f1b74152cb0 0x7f1b740c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 -- 172.21.15.129:0/47989158 >> 172.21.15.129:0/47989158 conn(0x7f1b740aa740 msgr2=0x7f1b740aab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 -- 172.21.15.129:0/47989158 shutdown_connections 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.582+0000 7f1b7a672640 1 -- 172.21.15.129:0/47989158 wait complete. 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.583+0000 7f1b7a672640 1 Processor -- start 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 -- start start 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b7414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b741eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74152cb0 0x7f1b7414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740c4920 con 0x7f1b740c8480 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1b740c3390 con 0x7f1b74152cb0 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b7a672640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1b740c4c00 con 0x7f1b740c3930 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b741eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b737fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b741eaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45422/0 (socket says 172.21.15.129:45422) 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.584+0000 7f1b737fe640 1 -- 172.21.15.129:0/938134752 learned_addr learned my addr 172.21.15.129:0/938134752 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b78c0f640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74152cb0 0x7f1b7414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b73fff640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b7414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b737fe640 1 -- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 msgr2=0x7f1b7414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b737fe640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b7414dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b737fe640 1 -- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74152cb0 msgr2=0x7f1b7414fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b737fe640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74152cb0 0x7f1b7414fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:54.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b737fe640 1 -- 172.21.15.129:0/938134752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7414dad0 con 0x7f1b740c8480 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b78c0f640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74152cb0 0x7f1b7414fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b737fe640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b741eaca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b6402a220 tx=0x7f1b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64037040 con 0x7f1b740c8480 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64042070 con 0x7f1b740c8480 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b74156830 con 0x7f1b740c8480 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64030bb0 con 0x7f1b740c8480 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.585+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741ee210 con 0x7f1b740c8480 2024-04-25T01:01:54.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.586+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b74075550 con 0x7f1b740c8480 2024-04-25T01:01:54.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.586+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b64047460 con 0x7f1b740c8480 2024-04-25T01:01:54.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.587+0000 7f1b70ff9640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1b5406be30 0x7f1b5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:54.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.587+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1b640abdb0 con 0x7f1b740c8480 2024-04-25T01:01:54.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.587+0000 7f1b73fff640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1b5406be30 0x7f1b5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:54.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.588+0000 7f1b73fff640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1b5406be30 0x7f1b5406e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b741505f0 tx=0x7f1b5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:54.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.591+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b64045360 con 0x7f1b740c8480 2024-04-25T01:01:54.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:54.801+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1b74068630 con 0x7f1b740c8480 2024-04-25T01:01:55.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.470+0000 7f1b70ff9640 1 -- 172.21.15.129:0/938134752 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f1b6407a490 con 0x7f1b740c8480 2024-04-25T01:01:55.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1b5406be30 msgr2=0x7f1b5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1b5406be30 0x7f1b5406e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1b741505f0 tx=0x7f1b5c008040 comp rx=0 tx=0).stop 2024-04-25T01:01:55.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 msgr2=0x7f1b741eaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b741eaca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1b6402a220 tx=0x7f1b640046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 shutdown_connections 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1b5406be30 0x7f1b5406e2f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b74152cb0 0x7f1b7414fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b740c8480 0x7f1b741eaca0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 --2- 172.21.15.129:0/938134752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1b740c3930 0x7f1b7414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 >> 172.21.15.129:0/938134752 conn(0x7f1b740aa740 msgr2=0x7f1b7406e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 shutdown_connections 2024-04-25T01:01:55.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.473+0000 7f1b7a672640 1 -- 172.21.15.129:0/938134752 wait complete. 2024-04-25T01:01:55.485 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T01:01:55.485 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:01:55.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.582+0000 7fbb91309640 1 Processor -- start 2024-04-25T01:01:55.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.582+0000 7fbb91309640 1 -- start start 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb91309640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 0x7fbb8c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb91309640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8c0c9860 con 0x7fbb8c152fa0 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb91309640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8c156690 con 0x7fbb8c1561e0 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb91309640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8c0c5a70 con 0x7fbb8c0c5520 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb8affd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fbb8c152fa0 0x7fbb8c0c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:57358/0 (socket says 172.21.15.129:57358) 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb8affd640 1 -- 172.21.15.129:0/3248758899 learned_addr learned my addr 172.21.15.129:0/3248758899 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb8a7fc640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 0x7fbb8c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.583+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870079336 0 0) 0x7fbb7c001490 con 0x7fbb8c152fa0 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb74003520 con 0x7fbb8c152fa0 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4229217069 0 0) 0x7fbb74003520 con 0x7fbb8c152fa0 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 msgr2=0x7fbb8c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 0x7fbb8c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 >> v1:172.21.15.171:6789/0 conn(0x7fbb8c1561e0 legacy=0x7fbb8c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c0c5d50 con 0x7fbb8c152fa0 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7fbb7c001bc0 con 0x7fbb8c152fa0 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 >> v1:172.21.15.129:6789/0 conn(0x7fbb8c152fa0 legacy=0x7fbb8c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb740039e0 0x7fbb74003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb74004800 0x7fbb74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 0x7fbb74005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c0c5a70 con 0x7fbb740039e0 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7400bb50 con 0x7fbb74004800 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7400bd20 con 0x7fbb8c0c5520 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8affd640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb740039e0 0x7fbb74003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8a7fc640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb74004800 0x7fbb74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8b7fe640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 0x7fbb74005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8affd640 1 -- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 msgr2=0x7fbb74005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8affd640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c5520 0x7fbb74005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8affd640 1 -- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb74004800 msgr2=0x7fbb74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8affd640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb74004800 0x7fbb74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.584+0000 7fbb8affd640 1 -- 172.21.15.129:0/3248758899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb74003520 con 0x7fbb740039e0 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb8affd640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb740039e0 0x7fbb74003dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00a9e0 tx=0x7fbb74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20272784d48ade14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00a600 con 0x7fbb740039e0 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb7c02f070 con 0x7fbb740039e0 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb89ffb640 1 -- 172.21.15.129:0/3248758899 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c02be60 con 0x7fbb740039e0 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb91309640 1 -- 172.21.15.129:0/3248758899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb740039e0 msgr2=0x7fbb74003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb91309640 1 --2- 172.21.15.129:0/3248758899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb740039e0 0x7fbb74003dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00a9e0 tx=0x7fbb74005830 comp rx=0 tx=0).stop 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb8b7fe640 1 -- 172.21.15.129:0/3248758899 reap_dead start 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb91309640 1 -- 172.21.15.129:0/3248758899 shutdown_connections 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb91309640 1 -- 172.21.15.129:0/3248758899 >> 172.21.15.129:0/3248758899 conn(0x7fbb8c06c400 msgr2=0x7fbb8c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb91309640 1 -- 172.21.15.129:0/3248758899 shutdown_connections 2024-04-25T01:01:55.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.585+0000 7fbb91309640 1 -- 172.21.15.129:0/3248758899 wait complete. 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 Processor -- start 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 -- start start 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb740039e0 0x7fbb8c1eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c4780 0x7fbb8c1e1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 0x7fbb8c1e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c0c93d0 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c0c6030 con 0x7fbb740039e0 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb91309640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8c0c9550 con 0x7fbb8c0c4780 2024-04-25T01:01:55.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb8b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 0x7fbb8c1e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb8b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 0x7fbb8c1e1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45438/0 (socket says 172.21.15.129:45438) 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.587+0000 7fbb8b7fe640 1 -- 172.21.15.129:0/218325572 learned_addr learned my addr 172.21.15.129:0/218325572 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8affd640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb740039e0 0x7fbb8c1eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8a7fc640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c4780 0x7fbb8c1e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8b7fe640 1 -- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c4780 msgr2=0x7fbb8c1e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8b7fe640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c4780 0x7fbb8c1e1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8b7fe640 1 -- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb740039e0 msgr2=0x7fbb8c1eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8b7fe640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb740039e0 0x7fbb8c1eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8b7fe640 1 -- 172.21.15.129:0/218325572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c0c1a20 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8affd640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb740039e0 0x7fbb8c1eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb8b7fe640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 0x7fbb8c1e1ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c1565c0 tx=0x7fbb8400e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84019070 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb840052e0 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84003810 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8c1ef6c0 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.588+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8c1ecd30 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.589+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c1ec840 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.589+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb8401c070 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.590+0000 7fbb737fe640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbb7806be30 0x7fbb7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.590+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb8408a6e0 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.590+0000 7fbb8affd640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbb7806be30 0x7fbb7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.590+0000 7fbb8affd640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbb7806be30 0x7fbb7806e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c033000 tx=0x7fbb7c038040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:55.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.594+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb84058d50 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.800+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb8c075550 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.801+0000 7fbb737fe640 1 -- 172.21.15.129:0/218325572 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fbb84058650 con 0x7fbb8c1ef0f0 2024-04-25T01:01:55.801 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:55.801 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:54.802314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:01:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-25T01:01:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbb7806be30 msgr2=0x7fbb7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbb7806be30 0x7fbb7806e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c033000 tx=0x7fbb7c038040 comp rx=0 tx=0).stop 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 msgr2=0x7fbb8c1e1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 0x7fbb8c1e1ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c1565c0 tx=0x7fbb8400e8a0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 shutdown_connections 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbb7806be30 0x7fbb7806e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb8c0c4780 0x7fbb8c1e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb8c1ef0f0 0x7fbb8c1e1ed0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 --2- 172.21.15.129:0/218325572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb740039e0 0x7fbb8c1eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 >> 172.21.15.129:0/218325572 conn(0x7fbb8c06c400 msgr2=0x7fbb8c0ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 shutdown_connections 2024-04-25T01:01:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.803+0000 7fbb91309640 1 -- 172.21.15.129:0/218325572 wait complete. 2024-04-25T01:01:55.814 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 Processor -- start 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 -- start start 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f240c150850 0x7f240c071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240c150c30 con 0x7f240c0671c0 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f240c0c1bf0 con 0x7f240c150850 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.910+0000 7f24110b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f240c0c2a70 con 0x7f240c06eba0 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240ad9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f240c150850 0x7f240c071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240a59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f240c0671c0 0x7f240c0c2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:57372/0 (socket says 172.21.15.129:57372) 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240a59c640 1 -- 172.21.15.129:0/2110787014 learned_addr learned my addr 172.21.15.129:0/2110787014 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240b59e640 1 --2- 172.21.15.129:0/2110787014 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240ad9d640 1 -- 172.21.15.129:0/2110787014 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 msgr2=0x7f240c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240ad9d640 1 --2- 172.21.15.129:0/2110787014 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240ad9d640 1 -- 172.21.15.129:0/2110787014 >> v1:172.21.15.129:6789/0 conn(0x7f240c0671c0 legacy=0x7f240c0c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f240ad9d640 1 -- 172.21.15.129:0/2110787014 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c0c2d50 con 0x7f240c150850 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.911+0000 7f2409d9b640 1 -- 172.21.15.129:0/2110787014 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946420174 0 0) 0x7f240c150c30 con 0x7f240c0671c0 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f240ad9d640 1 --2- 172.21.15.129:0/2110787014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c150850 0x7f240c071d30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0097a0 tx=0x7f23fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8925e17487542166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f2409d9b640 1 -- 172.21.15.129:0/2110787014 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc037070 con 0x7f240c150850 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f2409d9b640 1 -- 172.21.15.129:0/2110787014 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc032440 con 0x7f240c150850 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f2409d9b640 1 -- 172.21.15.129:0/2110787014 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc032810 con 0x7f240c150850 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 -- 172.21.15.129:0/2110787014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c150850 msgr2=0x7f240c071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 --2- 172.21.15.129:0/2110787014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c150850 0x7f240c071d30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0097a0 tx=0x7f23fc02a180 comp rx=0 tx=0).stop 2024-04-25T01:01:55.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 -- 172.21.15.129:0/2110787014 shutdown_connections 2024-04-25T01:01:55.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 --2- 172.21.15.129:0/2110787014 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 --2- 172.21.15.129:0/2110787014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c150850 0x7f240c071d30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 -- 172.21.15.129:0/2110787014 >> 172.21.15.129:0/2110787014 conn(0x7f240c0c71f0 msgr2=0x7f240c0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:55.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 -- 172.21.15.129:0/2110787014 shutdown_connections 2024-04-25T01:01:55.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.912+0000 7f24110b3640 1 -- 172.21.15.129:0/2110787014 wait complete. 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 Processor -- start 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 -- start start 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 0x7f240c13ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c150850 0x7f240c142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f240c06c8e0 con 0x7f240c0671c0 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f240c0c3310 con 0x7f240c150850 2024-04-25T01:01:55.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f24110b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f240c0c2790 con 0x7f240c06eba0 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f240ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 0x7f240c13ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f240ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 0x7f240c13ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45450/0 (socket says 172.21.15.129:45450) 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.914+0000 7f240ad9d640 1 -- 172.21.15.129:0/520783880 learned_addr learned my addr 172.21.15.129:0/520783880 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240a59c640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240ad9d640 1 -- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 msgr2=0x7f240c142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240ad9d640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240ad9d640 1 -- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c150850 msgr2=0x7f240c142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240ad9d640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c150850 0x7f240c142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240ad9d640 1 -- 172.21.15.129:0/520783880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c145eb0 con 0x7f240c0671c0 2024-04-25T01:01:55.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240a59c640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f240ad9d640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 0x7f240c13ff70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0097a0 tx=0x7f23fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc037050 con 0x7f240c0671c0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc038040 con 0x7f240c0671c0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c1409d0 con 0x7f240c0671c0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc004470 con 0x7f240c0671c0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.915+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c143930 con 0x7f240c0671c0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.916+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c14fa50 con 0x7f240c0671c0 2024-04-25T01:01:55.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.917+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23fc03c6d0 con 0x7f240c0671c0 2024-04-25T01:01:55.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.917+0000 7f23f37fe640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f23f806be30 0x7f23f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:55.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.917+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f23fc0aba40 con 0x7f240c0671c0 2024-04-25T01:01:55.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.917+0000 7f240a59c640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f23f806be30 0x7f23f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:55.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.918+0000 7f240a59c640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f23f806be30 0x7f23f806e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f240c14e630 tx=0x7f2400009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:55.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:55.921+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f240c14fa50 con 0x7f240c0671c0 2024-04-25T01:01:56.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.095+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f240c075550 con 0x7f240c0671c0 2024-04-25T01:01:56.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.096+0000 7f23f37fe640 1 -- 172.21.15.129:0/520783880 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f23fc0abe20 con 0x7f240c0671c0 2024-04-25T01:01:56.097 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:56.097 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:56.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f23f806be30 msgr2=0x7f23f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f23f806be30 0x7f23f806e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f240c14e630 tx=0x7f2400009040 comp rx=0 tx=0).stop 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 msgr2=0x7f240c13ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 0x7f240c13ff70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0097a0 tx=0x7f23fc0046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 shutdown_connections 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f23f806be30 0x7f23f806e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f240c06eba0 0x7f240c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f240c150850 0x7f240c142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 --2- 172.21.15.129:0/520783880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f240c0671c0 0x7f240c13ff70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 >> 172.21.15.129:0/520783880 conn(0x7f240c0c71f0 msgr2=0x7f240c14f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 shutdown_connections 2024-04-25T01:01:56.100 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.099+0000 7f24110b3640 1 -- 172.21.15.129:0/520783880 wait complete. 2024-04-25T01:01:56.111 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:01:56.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:01:56.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.206+0000 7f8434a6d640 1 Processor -- start 2024-04-25T01:01:56.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.206+0000 7f8434a6d640 1 -- start start 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8434a6d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f8430152bc0 0x7f84300c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8434a6d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f8430159300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8434a6d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f843006cb10 0x7f843006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8434a6d640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f843006c680 con 0x7f8430152bc0 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8434a6d640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f843006e780 con 0x7f843006cb10 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8434a6d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f8430159840 con 0x7f84300c0470 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f8430152bc0 0x7f84300c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f8430152bc0 0x7f84300c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45452/0 (socket says 172.21.15.129:45452) 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 -- 172.21.15.129:0/2906190339 learned_addr learned my addr 172.21.15.129:0/2906190339 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f8427fff640 1 --2- 172.21.15.129:0/2906190339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f8430159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842ed9d640 1 --2- 172.21.15.129:0/2906190339 >> v2:172.21.15.171:3300/0 conn(0x7f843006cb10 0x7f843006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 -- 172.21.15.129:0/2906190339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 msgr2=0x7f8430159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 --2- 172.21.15.129:0/2906190339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f8430159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 -- 172.21.15.129:0/2906190339 >> v2:172.21.15.171:3300/0 conn(0x7f843006cb10 msgr2=0x7f843006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 --2- 172.21.15.129:0/2906190339 >> v2:172.21.15.171:3300/0 conn(0x7f843006cb10 0x7f843006d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.207+0000 7f842e59c640 1 -- 172.21.15.129:0/2906190339 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84300c0970 con 0x7f8430152bc0 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f842e59c640 1 --2- 172.21.15.129:0/2906190339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8430152bc0 0x7f84300c20d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84300a5f20 tx=0x7f842002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396b12ff63acfa8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f842dd9b640 1 -- 172.21.15.129:0/2906190339 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420037070 con 0x7f8430152bc0 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f842dd9b640 1 -- 172.21.15.129:0/2906190339 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8420032440 con 0x7f8430152bc0 2024-04-25T01:01:56.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f842dd9b640 1 -- 172.21.15.129:0/2906190339 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420032810 con 0x7f8430152bc0 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 -- 172.21.15.129:0/2906190339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8430152bc0 msgr2=0x7f84300c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 --2- 172.21.15.129:0/2906190339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8430152bc0 0x7f84300c20d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84300a5f20 tx=0x7f842002a1e0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 -- 172.21.15.129:0/2906190339 shutdown_connections 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 --2- 172.21.15.129:0/2906190339 >> v2:172.21.15.171:3300/0 conn(0x7f843006cb10 0x7f843006d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 --2- 172.21.15.129:0/2906190339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f8430159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 --2- 172.21.15.129:0/2906190339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8430152bc0 0x7f84300c20d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 -- 172.21.15.129:0/2906190339 >> 172.21.15.129:0/2906190339 conn(0x7f84300a86b0 msgr2=0x7f84300a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 -- 172.21.15.129:0/2906190339 shutdown_connections 2024-04-25T01:01:56.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.208+0000 7f8434a6d640 1 -- 172.21.15.129:0/2906190339 wait complete. 2024-04-25T01:01:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 Processor -- start 2024-04-25T01:01:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 -- start start 2024-04-25T01:01:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 0x7f843014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f84301531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8430152bc0 0x7f84300715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84300c0f30 con 0x7f843006cb10 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f843006c310 con 0x7f8430152bc0 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f8434a6d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f843006cfa0 con 0x7f84300c0470 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f842e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 0x7f843014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f842e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 0x7f843014b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45468/0 (socket says 172.21.15.129:45468) 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.210+0000 7f842e59c640 1 -- 172.21.15.129:0/1941614318 learned_addr learned my addr 172.21.15.129:0/1941614318 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f8427fff640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f84301531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842ed9d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8430152bc0 0x7f84300715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842e59c640 1 -- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 msgr2=0x7f84301531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842e59c640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f84301531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842e59c640 1 -- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8430152bc0 msgr2=0x7f84300715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842e59c640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8430152bc0 0x7f84300715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842e59c640 1 -- 172.21.15.129:0/1941614318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84301551e0 con 0x7f843006cb10 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842ed9d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8430152bc0 0x7f84300715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:56.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f842e59c640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 0x7f843014b6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f84200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420041020 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84200306c0 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84301554c0 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420037040 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.211+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8430155a80 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.212+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8430075550 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.212+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8420044020 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.213+0000 7f8426ffd640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f841006be30 0x7f841006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.213+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f84200ab880 con 0x7f843006cb10 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.213+0000 7f8427fff640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f841006be30 0x7f841006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.213+0000 7f8427fff640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f841006be30 0x7f841006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f84300c2ee0 tx=0x7f8418008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.217+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f842004b030 con 0x7f843006cb10 2024-04-25T01:01:56.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.386+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84301eb900 con 0x7f841006be30 2024-04-25T01:01:56.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.395+0000 7f8426ffd640 1 -- 172.21.15.129:0/1941614318 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f84301eb900 con 0x7f841006be30 2024-04-25T01:01:56.400 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:56.402 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:01:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f841006be30 msgr2=0x7f841006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f841006be30 0x7f841006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f84300c2ee0 tx=0x7f8418008040 comp rx=0 tx=0).stop 2024-04-25T01:01:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 msgr2=0x7f843014b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 0x7f843014b6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f84200046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 shutdown_connections 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f841006be30 0x7f841006e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84300c0470 0x7f84301531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8430152bc0 0x7f84300715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 --2- 172.21.15.129:0/1941614318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f843006cb10 0x7f843014b6b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 >> 172.21.15.129:0/1941614318 conn(0x7f84300a86b0 msgr2=0x7f84300a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 shutdown_connections 2024-04-25T01:01:56.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.399+0000 7f8434a6d640 1 -- 172.21.15.129:0/1941614318 wait complete. 2024-04-25T01:01:56.411 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T01:01:54.670354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219492,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755228,"statfs":{"total":773094113280,"available":772869353472,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468886+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:01:39.468886+0000","last_peered":"2024-04-25T01:01:39.468886+0000","last_clean":"2024-04-25T01:01:39.468886+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:01:39.468886+0000","last_undegraded":"2024-04-25T01:01:39.468886+0000","last_fullsized":"2024-04-25T01:01:39.468886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415373+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:01:38.415373+0000","last_peered":"2024-04-25T01:01:38.415373+0000","last_clean":"2024-04-25T01:01:38.415373+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:01:38.415373+0000","last_undegraded":"2024-04-25T01:01:38.415373+0000","last_fullsized":"2024-04-25T01:01:38.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414835+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:01:38.414835+0000","last_peered":"2024-04-25T01:01:38.414835+0000","last_clean":"2024-04-25T01:01:38.414835+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:01:38.414835+0000","last_undegraded":"2024-04-25T01:01:38.414835+0000","last_fullsized":"2024-04-25T01:01:38.414835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468718+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:01:39.468718+0000","last_peered":"2024-04-25T01:01:39.468718+0000","last_clean":"2024-04-25T01:01:39.468718+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:01:39.468718+0000","last_undegraded":"2024-04-25T01:01:39.468718+0000","last_fullsized":"2024-04-25T01:01:39.468718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415475+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:01:38.415475+0000","last_peered":"2024-04-25T01:01:38.415475+0000","last_clean":"2024-04-25T01:01:38.415475+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:01:38.415475+0000","last_undegraded":"2024-04-25T01:01:38.415475+0000","last_fullsized":"2024-04-25T01:01:38.415475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417116+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:01:38.417116+0000","last_peered":"2024-04-25T01:01:38.417116+0000","last_clean":"2024-04-25T01:01:38.417116+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:01:38.417116+0000","last_undegraded":"2024-04-25T01:01:38.417116+0000","last_fullsized":"2024-04-25T01:01:38.417116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459319+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:01:38.459319+0000","last_peered":"2024-04-25T01:01:38.459319+0000","last_clean":"2024-04-25T01:01:38.459319+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:01:38.459319+0000","last_undegraded":"2024-04-25T01:01:38.459319+0000","last_fullsized":"2024-04-25T01:01:38.459319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416961+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:01:38.416961+0000","last_peered":"2024-04-25T01:01:38.416961+0000","last_clean":"2024-04-25T01:01:38.416961+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:01:38.416961+0000","last_undegraded":"2024-04-25T01:01:38.416961+0000","last_fullsized":"2024-04-25T01:01:38.416961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416812+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:01:38.416812+0000","last_peered":"2024-04-25T01:01:38.416812+0000","last_clean":"2024-04-25T01:01:38.416812+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:01:38.416812+0000","last_undegraded":"2024-04-25T01:01:38.416812+0000","last_fullsized":"2024-04-25T01:01:38.416812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459435+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:01:38.459435+0000","last_peered":"2024-04-25T01:01:38.459435+0000","last_clean":"2024-04-25T01:01:38.459435+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:01:38.459435+0000","last_undegraded":"2024-04-25T01:01:38.459435+0000","last_fullsized":"2024-04-25T01:01:38.459435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416306+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:01:38.416306+0000","last_peered":"2024-04-25T01:01:38.416306+0000","last_clean":"2024-04-25T01:01:38.416306+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:01:38.416306+0000","last_undegraded":"2024-04-25T01:01:38.416306+0000","last_fullsized":"2024-04-25T01:01:38.416306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097726+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:01:39.097726+0000","last_peered":"2024-04-25T01:01:39.097726+0000","last_clean":"2024-04-25T01:01:39.097726+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:01:39.097726+0000","last_undegraded":"2024-04-25T01:01:39.097726+0000","last_fullsized":"2024-04-25T01:01:39.097726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416250+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:01:38.416250+0000","last_peered":"2024-04-25T01:01:38.416250+0000","last_clean":"2024-04-25T01:01:38.416250+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:01:38.416250+0000","last_undegraded":"2024-04-25T01:01:38.416250+0000","last_fullsized":"2024-04-25T01:01:38.416250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416610+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:01:38.416610+0000","last_peered":"2024-04-25T01:01:38.416610+0000","last_clean":"2024-04-25T01:01:38.416610+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:01:38.416610+0000","last_undegraded":"2024-04-25T01:01:38.416610+0000","last_fullsized":"2024-04-25T01:01:38.416610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415556+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:01:38.415556+0000","last_peered":"2024-04-25T01:01:38.415556+0000","last_clean":"2024-04-25T01:01:38.415556+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:01:38.415556+0000","last_undegraded":"2024-04-25T01:01:38.415556+0000","last_fullsized":"2024-04-25T01:01:38.415556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551986+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:01:38.551986+0000","last_peered":"2024-04-25T01:01:38.551986+0000","last_clean":"2024-04-25T01:01:38.551986+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:01:38.551986+0000","last_undegraded":"2024-04-25T01:01:38.551986+0000","last_fullsized":"2024-04-25T01:01:38.551986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415994+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:01:38.415994+0000","last_peered":"2024-04-25T01:01:38.415994+0000","last_clean":"2024-04-25T01:01:38.415994+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:01:38.415994+0000","last_undegraded":"2024-04-25T01:01:38.415994+0000","last_fullsized":"2024-04-25T01:01:38.415994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415087+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:01:38.415087+0000","last_peered":"2024-04-25T01:01:38.415087+0000","last_clean":"2024-04-25T01:01:38.415087+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:01:38.415087+0000","last_undegraded":"2024-04-25T01:01:38.415087+0000","last_fullsized":"2024-04-25T01:01:38.415087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551639+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:01:38.551639+0000","last_peered":"2024-04-25T01:01:38.551639+0000","last_clean":"2024-04-25T01:01:38.551639+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:01:38.551639+0000","last_undegraded":"2024-04-25T01:01:38.551639+0000","last_fullsized":"2024-04-25T01:01:38.551639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552136+0000","last_change":"2024-04-25T01:01:37.436247+0000","last_active":"2024-04-25T01:01:38.552136+0000","last_peered":"2024-04-25T01:01:38.552136+0000","last_clean":"2024-04-25T01:01:38.552136+0000","last_became_active":"2024-04-25T01:01:37.435388+0000","last_became_peered":"2024-04-25T01:01:37.435388+0000","last_unstale":"2024-04-25T01:01:38.552136+0000","last_undegraded":"2024-04-25T01:01:38.552136+0000","last_fullsized":"2024-04-25T01:01:38.552136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416152+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:01:38.416152+0000","last_peered":"2024-04-25T01:01:38.416152+0000","last_clean":"2024-04-25T01:01:38.416152+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:01:38.416152+0000","last_undegraded":"2024-04-25T01:01:38.416152+0000","last_fullsized":"2024-04-25T01:01:38.416152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415346+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:01:38.415346+0000","last_peered":"2024-04-25T01:01:38.415346+0000","last_clean":"2024-04-25T01:01:38.415346+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:01:38.415346+0000","last_undegraded":"2024-04-25T01:01:38.415346+0000","last_fullsized":"2024-04-25T01:01:38.415346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416144+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:01:38.416144+0000","last_peered":"2024-04-25T01:01:38.416144+0000","last_clean":"2024-04-25T01:01:38.416144+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:01:38.416144+0000","last_undegraded":"2024-04-25T01:01:38.416144+0000","last_fullsized":"2024-04-25T01:01:38.416144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468858+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:01:39.468858+0000","last_peered":"2024-04-25T01:01:39.468858+0000","last_clean":"2024-04-25T01:01:39.468858+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:01:39.468858+0000","last_undegraded":"2024-04-25T01:01:39.468858+0000","last_fullsized":"2024-04-25T01:01:39.468858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098463+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:01:39.098463+0000","last_peered":"2024-04-25T01:01:39.098463+0000","last_clean":"2024-04-25T01:01:39.098463+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:01:39.098463+0000","last_undegraded":"2024-04-25T01:01:39.098463+0000","last_fullsized":"2024-04-25T01:01:39.098463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415860+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:01:38.415860+0000","last_peered":"2024-04-25T01:01:38.415860+0000","last_clean":"2024-04-25T01:01:38.415860+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:01:38.415860+0000","last_undegraded":"2024-04-25T01:01:38.415860+0000","last_fullsized":"2024-04-25T01:01:38.415860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415706+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:01:38.415706+0000","last_peered":"2024-04-25T01:01:38.415706+0000","last_clean":"2024-04-25T01:01:38.415706+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:01:38.415706+0000","last_undegraded":"2024-04-25T01:01:38.415706+0000","last_fullsized":"2024-04-25T01:01:38.415706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552159+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:01:38.552159+0000","last_peered":"2024-04-25T01:01:38.552159+0000","last_clean":"2024-04-25T01:01:38.552159+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:01:38.552159+0000","last_undegraded":"2024-04-25T01:01:38.552159+0000","last_fullsized":"2024-04-25T01:01:38.552159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414731+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:01:38.414731+0000","last_peered":"2024-04-25T01:01:38.414731+0000","last_clean":"2024-04-25T01:01:38.414731+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:01:38.414731+0000","last_undegraded":"2024-04-25T01:01:38.414731+0000","last_fullsized":"2024-04-25T01:01:38.414731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552149+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:01:38.552149+0000","last_peered":"2024-04-25T01:01:38.552149+0000","last_clean":"2024-04-25T01:01:38.552149+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:01:38.552149+0000","last_undegraded":"2024-04-25T01:01:38.552149+0000","last_fullsized":"2024-04-25T01:01:38.552149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417245+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:01:38.417245+0000","last_peered":"2024-04-25T01:01:38.417245+0000","last_clean":"2024-04-25T01:01:38.417245+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:01:38.417245+0000","last_undegraded":"2024-04-25T01:01:38.417245+0000","last_fullsized":"2024-04-25T01:01:38.417245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468995+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:01:39.468995+0000","last_peered":"2024-04-25T01:01:39.468995+0000","last_clean":"2024-04-25T01:01:39.468995+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:01:39.468995+0000","last_undegraded":"2024-04-25T01:01:39.468995+0000","last_fullsized":"2024-04-25T01:01:39.468995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459566+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:01:38.459566+0000","last_peered":"2024-04-25T01:01:38.459566+0000","last_clean":"2024-04-25T01:01:38.459566+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:01:38.459566+0000","last_undegraded":"2024-04-25T01:01:38.459566+0000","last_fullsized":"2024-04-25T01:01:38.459566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552153+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:01:38.552153+0000","last_peered":"2024-04-25T01:01:38.552153+0000","last_clean":"2024-04-25T01:01:38.552153+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:01:38.552153+0000","last_undegraded":"2024-04-25T01:01:38.552153+0000","last_fullsized":"2024-04-25T01:01:38.552153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551709+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:01:38.551709+0000","last_peered":"2024-04-25T01:01:38.551709+0000","last_clean":"2024-04-25T01:01:38.551709+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:01:38.551709+0000","last_undegraded":"2024-04-25T01:01:38.551709+0000","last_fullsized":"2024-04-25T01:01:38.551709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416998+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:01:38.416998+0000","last_peered":"2024-04-25T01:01:38.416998+0000","last_clean":"2024-04-25T01:01:38.416998+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:01:38.416998+0000","last_undegraded":"2024-04-25T01:01:38.416998+0000","last_fullsized":"2024-04-25T01:01:38.416998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459358+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:01:38.459358+0000","last_peered":"2024-04-25T01:01:38.459358+0000","last_clean":"2024-04-25T01:01:38.459358+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:01:38.459358+0000","last_undegraded":"2024-04-25T01:01:38.459358+0000","last_fullsized":"2024-04-25T01:01:38.459358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552085+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:01:38.552085+0000","last_peered":"2024-04-25T01:01:38.552085+0000","last_clean":"2024-04-25T01:01:38.552085+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:01:38.552085+0000","last_undegraded":"2024-04-25T01:01:38.552085+0000","last_fullsized":"2024-04-25T01:01:38.552085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551641+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:01:38.551641+0000","last_peered":"2024-04-25T01:01:38.551641+0000","last_clean":"2024-04-25T01:01:38.551641+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:01:38.551641+0000","last_undegraded":"2024-04-25T01:01:38.551641+0000","last_fullsized":"2024-04-25T01:01:38.551641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575958+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:01:39.575958+0000","last_peered":"2024-04-25T01:01:39.575958+0000","last_clean":"2024-04-25T01:01:39.575958+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:01:39.575958+0000","last_undegraded":"2024-04-25T01:01:39.575958+0000","last_fullsized":"2024-04-25T01:01:39.575958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552170+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:01:38.552170+0000","last_peered":"2024-04-25T01:01:38.552170+0000","last_clean":"2024-04-25T01:01:38.552170+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:01:38.552170+0000","last_undegraded":"2024-04-25T01:01:38.552170+0000","last_fullsized":"2024-04-25T01:01:38.552170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468365+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:01:39.468365+0000","last_peered":"2024-04-25T01:01:39.468365+0000","last_clean":"2024-04-25T01:01:39.468365+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:01:39.468365+0000","last_undegraded":"2024-04-25T01:01:39.468365+0000","last_fullsized":"2024-04-25T01:01:39.468365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415600+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:01:38.415600+0000","last_peered":"2024-04-25T01:01:38.415600+0000","last_clean":"2024-04-25T01:01:38.415600+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:01:38.415600+0000","last_undegraded":"2024-04-25T01:01:38.415600+0000","last_fullsized":"2024-04-25T01:01:38.415600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468734+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:01:39.468734+0000","last_peered":"2024-04-25T01:01:39.468734+0000","last_clean":"2024-04-25T01:01:39.468734+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:01:39.468734+0000","last_undegraded":"2024-04-25T01:01:39.468734+0000","last_fullsized":"2024-04-25T01:01:39.468734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414950+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:01:38.414950+0000","last_peered":"2024-04-25T01:01:38.414950+0000","last_clean":"2024-04-25T01:01:38.414950+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:01:38.414950+0000","last_undegraded":"2024-04-25T01:01:38.414950+0000","last_fullsized":"2024-04-25T01:01:38.414950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459194+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:01:38.459194+0000","last_peered":"2024-04-25T01:01:38.459194+0000","last_clean":"2024-04-25T01:01:38.459194+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:01:38.459194+0000","last_undegraded":"2024-04-25T01:01:38.459194+0000","last_fullsized":"2024-04-25T01:01:38.459194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415719+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:01:38.415719+0000","last_peered":"2024-04-25T01:01:38.415719+0000","last_clean":"2024-04-25T01:01:38.415719+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:01:38.415719+0000","last_undegraded":"2024-04-25T01:01:38.415719+0000","last_fullsized":"2024-04-25T01:01:38.415719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098131+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:01:39.098131+0000","last_peered":"2024-04-25T01:01:39.098131+0000","last_clean":"2024-04-25T01:01:39.098131+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:01:39.098131+0000","last_undegraded":"2024-04-25T01:01:39.098131+0000","last_fullsized":"2024-04-25T01:01:39.098131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417023+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:01:38.417023+0000","last_peered":"2024-04-25T01:01:38.417023+0000","last_clean":"2024-04-25T01:01:38.417023+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:01:38.417023+0000","last_undegraded":"2024-04-25T01:01:38.417023+0000","last_fullsized":"2024-04-25T01:01:38.417023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576180+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:01:39.576180+0000","last_peered":"2024-04-25T01:01:39.576180+0000","last_clean":"2024-04-25T01:01:39.576180+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:01:39.576180+0000","last_undegraded":"2024-04-25T01:01:39.576180+0000","last_fullsized":"2024-04-25T01:01:39.576180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097822+0000","last_change":"2024-04-25T01:01:37.432081+0000","last_active":"2024-04-25T01:01:39.097822+0000","last_peered":"2024-04-25T01:01:39.097822+0000","last_clean":"2024-04-25T01:01:39.097822+0000","last_became_active":"2024-04-25T01:01:37.431657+0000","last_became_peered":"2024-04-25T01:01:37.431657+0000","last_unstale":"2024-04-25T01:01:39.097822+0000","last_undegraded":"2024-04-25T01:01:39.097822+0000","last_fullsized":"2024-04-25T01:01:39.097822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459452+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:01:38.459452+0000","last_peered":"2024-04-25T01:01:38.459452+0000","last_clean":"2024-04-25T01:01:38.459452+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:01:38.459452+0000","last_undegraded":"2024-04-25T01:01:38.459452+0000","last_fullsized":"2024-04-25T01:01:38.459452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552164+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:01:38.552164+0000","last_peered":"2024-04-25T01:01:38.552164+0000","last_clean":"2024-04-25T01:01:38.552164+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:01:38.552164+0000","last_undegraded":"2024-04-25T01:01:38.552164+0000","last_fullsized":"2024-04-25T01:01:38.552164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468921+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:01:39.468921+0000","last_peered":"2024-04-25T01:01:39.468921+0000","last_clean":"2024-04-25T01:01:39.468921+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:01:39.468921+0000","last_undegraded":"2024-04-25T01:01:39.468921+0000","last_fullsized":"2024-04-25T01:01:39.468921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417383+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:01:38.417383+0000","last_peered":"2024-04-25T01:01:38.417383+0000","last_clean":"2024-04-25T01:01:38.417383+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:01:38.417383+0000","last_undegraded":"2024-04-25T01:01:38.417383+0000","last_fullsized":"2024-04-25T01:01:38.417383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576094+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:01:39.576094+0000","last_peered":"2024-04-25T01:01:39.576094+0000","last_clean":"2024-04-25T01:01:39.576094+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:01:39.576094+0000","last_undegraded":"2024-04-25T01:01:39.576094+0000","last_fullsized":"2024-04-25T01:01:39.576094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458996+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:01:38.458996+0000","last_peered":"2024-04-25T01:01:38.458996+0000","last_clean":"2024-04-25T01:01:38.458996+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:01:38.458996+0000","last_undegraded":"2024-04-25T01:01:38.458996+0000","last_fullsized":"2024-04-25T01:01:38.458996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416863+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:01:38.416863+0000","last_peered":"2024-04-25T01:01:38.416863+0000","last_clean":"2024-04-25T01:01:38.416863+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:01:38.416863+0000","last_undegraded":"2024-04-25T01:01:38.416863+0000","last_fullsized":"2024-04-25T01:01:38.416863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416428+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:01:38.416428+0000","last_peered":"2024-04-25T01:01:38.416428+0000","last_clean":"2024-04-25T01:01:38.416428+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:01:38.416428+0000","last_undegraded":"2024-04-25T01:01:38.416428+0000","last_fullsized":"2024-04-25T01:01:38.416428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415619+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:01:38.415619+0000","last_peered":"2024-04-25T01:01:38.415619+0000","last_clean":"2024-04-25T01:01:38.415619+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:01:38.415619+0000","last_undegraded":"2024-04-25T01:01:38.415619+0000","last_fullsized":"2024-04-25T01:01:38.415619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415805+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:01:38.415805+0000","last_peered":"2024-04-25T01:01:38.415805+0000","last_clean":"2024-04-25T01:01:38.415805+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:01:38.415805+0000","last_undegraded":"2024-04-25T01:01:38.415805+0000","last_fullsized":"2024-04-25T01:01:38.415805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575808+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:01:39.575808+0000","last_peered":"2024-04-25T01:01:39.575808+0000","last_clean":"2024-04-25T01:01:39.575808+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:01:39.575808+0000","last_undegraded":"2024-04-25T01:01:39.575808+0000","last_fullsized":"2024-04-25T01:01:39.575808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468765+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:01:39.468765+0000","last_peered":"2024-04-25T01:01:39.468765+0000","last_clean":"2024-04-25T01:01:39.468765+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:01:39.468765+0000","last_undegraded":"2024-04-25T01:01:39.468765+0000","last_fullsized":"2024-04-25T01:01:39.468765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097771+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:01:39.097771+0000","last_peered":"2024-04-25T01:01:39.097771+0000","last_clean":"2024-04-25T01:01:39.097771+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:01:39.097771+0000","last_undegraded":"2024-04-25T01:01:39.097771+0000","last_fullsized":"2024-04-25T01:01:39.097771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097883+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:01:39.097883+0000","last_peered":"2024-04-25T01:01:39.097883+0000","last_clean":"2024-04-25T01:01:39.097883+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:01:39.097883+0000","last_undegraded":"2024-04-25T01:01:39.097883+0000","last_fullsized":"2024-04-25T01:01:39.097883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415652+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:01:38.415652+0000","last_peered":"2024-04-25T01:01:38.415652+0000","last_clean":"2024-04-25T01:01:38.415652+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:01:38.415652+0000","last_undegraded":"2024-04-25T01:01:38.415652+0000","last_fullsized":"2024-04-25T01:01:38.415652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416147+0000","last_change":"2024-04-25T01:01:33.389402+0000","last_active":"2024-04-25T01:01:38.416147+0000","last_peered":"2024-04-25T01:01:38.416147+0000","last_clean":"2024-04-25T01:01:38.416147+0000","last_became_active":"2024-04-25T01:01:33.389147+0000","last_became_peered":"2024-04-25T01:01:33.389147+0000","last_unstale":"2024-04-25T01:01:38.416147+0000","last_undegraded":"2024-04-25T01:01:38.416147+0000","last_fullsized":"2024-04-25T01:01:38.416147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459311+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:01:38.459311+0000","last_peered":"2024-04-25T01:01:38.459311+0000","last_clean":"2024-04-25T01:01:38.459311+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:01:38.459311+0000","last_undegraded":"2024-04-25T01:01:38.459311+0000","last_fullsized":"2024-04-25T01:01:38.459311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415085+0000","last_change":"2024-04-25T01:01:37.441477+0000","last_active":"2024-04-25T01:01:38.415085+0000","last_peered":"2024-04-25T01:01:38.415085+0000","last_clean":"2024-04-25T01:01:38.415085+0000","last_became_active":"2024-04-25T01:01:37.441099+0000","last_became_peered":"2024-04-25T01:01:37.441099+0000","last_unstale":"2024-04-25T01:01:38.415085+0000","last_undegraded":"2024-04-25T01:01:38.415085+0000","last_fullsized":"2024-04-25T01:01:38.415085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468846+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:01:39.468846+0000","last_peered":"2024-04-25T01:01:39.468846+0000","last_clean":"2024-04-25T01:01:39.468846+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:01:39.468846+0000","last_undegraded":"2024-04-25T01:01:39.468846+0000","last_fullsized":"2024-04-25T01:01:39.468846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469030+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:01:39.469030+0000","last_peered":"2024-04-25T01:01:39.469030+0000","last_clean":"2024-04-25T01:01:39.469030+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:01:39.469030+0000","last_undegraded":"2024-04-25T01:01:39.469030+0000","last_fullsized":"2024-04-25T01:01:39.469030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459574+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:01:38.459574+0000","last_peered":"2024-04-25T01:01:38.459574+0000","last_clean":"2024-04-25T01:01:38.459574+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:01:38.459574+0000","last_undegraded":"2024-04-25T01:01:38.459574+0000","last_fullsized":"2024-04-25T01:01:38.459574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416957+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:01:38.416957+0000","last_peered":"2024-04-25T01:01:38.416957+0000","last_clean":"2024-04-25T01:01:38.416957+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:01:38.416957+0000","last_undegraded":"2024-04-25T01:01:38.416957+0000","last_fullsized":"2024-04-25T01:01:38.416957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:43.083494+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:01:43.083494+0000","last_peered":"2024-04-25T01:01:43.083494+0000","last_clean":"2024-04-25T01:01:43.083494+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:01:43.083494+0000","last_undegraded":"2024-04-25T01:01:43.083494+0000","last_fullsized":"2024-04-25T01:01:43.083494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551940+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:01:38.551940+0000","last_peered":"2024-04-25T01:01:38.551940+0000","last_clean":"2024-04-25T01:01:38.551940+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:01:38.551940+0000","last_undegraded":"2024-04-25T01:01:38.551940+0000","last_fullsized":"2024-04-25T01:01:38.551940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416242+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:01:38.416242+0000","last_peered":"2024-04-25T01:01:38.416242+0000","last_clean":"2024-04-25T01:01:38.416242+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:01:38.416242+0000","last_undegraded":"2024-04-25T01:01:38.416242+0000","last_fullsized":"2024-04-25T01:01:38.416242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552062+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:01:38.552062+0000","last_peered":"2024-04-25T01:01:38.552062+0000","last_clean":"2024-04-25T01:01:38.552062+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:01:38.552062+0000","last_undegraded":"2024-04-25T01:01:38.552062+0000","last_fullsized":"2024-04-25T01:01:38.552062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416785+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:01:38.416785+0000","last_peered":"2024-04-25T01:01:38.416785+0000","last_clean":"2024-04-25T01:01:38.416785+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:01:38.416785+0000","last_undegraded":"2024-04-25T01:01:38.416785+0000","last_fullsized":"2024-04-25T01:01:38.416785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468828+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:01:39.468828+0000","last_peered":"2024-04-25T01:01:39.468828+0000","last_clean":"2024-04-25T01:01:39.468828+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:01:39.468828+0000","last_undegraded":"2024-04-25T01:01:39.468828+0000","last_fullsized":"2024-04-25T01:01:39.468828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459534+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:01:38.459534+0000","last_peered":"2024-04-25T01:01:38.459534+0000","last_clean":"2024-04-25T01:01:38.459534+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:01:38.459534+0000","last_undegraded":"2024-04-25T01:01:38.459534+0000","last_fullsized":"2024-04-25T01:01:38.459534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459526+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:01:38.459526+0000","last_peered":"2024-04-25T01:01:38.459526+0000","last_clean":"2024-04-25T01:01:38.459526+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:01:38.459526+0000","last_undegraded":"2024-04-25T01:01:38.459526+0000","last_fullsized":"2024-04-25T01:01:38.459526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414973+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:01:38.414973+0000","last_peered":"2024-04-25T01:01:38.414973+0000","last_clean":"2024-04-25T01:01:38.414973+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:01:38.414973+0000","last_undegraded":"2024-04-25T01:01:38.414973+0000","last_fullsized":"2024-04-25T01:01:38.414973+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468563+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:01:39.468563+0000","last_peered":"2024-04-25T01:01:39.468563+0000","last_clean":"2024-04-25T01:01:39.468563+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:01:39.468563+0000","last_undegraded":"2024-04-25T01:01:39.468563+0000","last_fullsized":"2024-04-25T01:01:39.468563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551523+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:01:38.551523+0000","last_peered":"2024-04-25T01:01:38.551523+0000","last_clean":"2024-04-25T01:01:38.551523+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:01:38.551523+0000","last_undegraded":"2024-04-25T01:01:38.551523+0000","last_fullsized":"2024-04-25T01:01:38.551523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459092+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:01:38.459092+0000","last_peered":"2024-04-25T01:01:38.459092+0000","last_clean":"2024-04-25T01:01:38.459092+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:01:38.459092+0000","last_undegraded":"2024-04-25T01:01:38.459092+0000","last_fullsized":"2024-04-25T01:01:38.459092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098352+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:01:39.098352+0000","last_peered":"2024-04-25T01:01:39.098352+0000","last_clean":"2024-04-25T01:01:39.098352+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:01:39.098352+0000","last_undegraded":"2024-04-25T01:01:39.098352+0000","last_fullsized":"2024-04-25T01:01:39.098352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414947+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:01:38.414947+0000","last_peered":"2024-04-25T01:01:38.414947+0000","last_clean":"2024-04-25T01:01:38.414947+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:01:38.414947+0000","last_undegraded":"2024-04-25T01:01:38.414947+0000","last_fullsized":"2024-04-25T01:01:38.414947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415843+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:01:38.415843+0000","last_peered":"2024-04-25T01:01:38.415843+0000","last_clean":"2024-04-25T01:01:38.415843+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:01:38.415843+0000","last_undegraded":"2024-04-25T01:01:38.415843+0000","last_fullsized":"2024-04-25T01:01:38.415843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097915+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:01:39.097915+0000","last_peered":"2024-04-25T01:01:39.097915+0000","last_clean":"2024-04-25T01:01:39.097915+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:01:39.097915+0000","last_undegraded":"2024-04-25T01:01:39.097915+0000","last_fullsized":"2024-04-25T01:01:39.097915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469001+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:01:39.469001+0000","last_peered":"2024-04-25T01:01:39.469001+0000","last_clean":"2024-04-25T01:01:39.469001+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:01:39.469001+0000","last_undegraded":"2024-04-25T01:01:39.469001+0000","last_fullsized":"2024-04-25T01:01:39.469001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576127+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:01:39.576127+0000","last_peered":"2024-04-25T01:01:39.576127+0000","last_clean":"2024-04-25T01:01:39.576127+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:01:39.576127+0000","last_undegraded":"2024-04-25T01:01:39.576127+0000","last_fullsized":"2024-04-25T01:01:39.576127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415723+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:01:38.415723+0000","last_peered":"2024-04-25T01:01:38.415723+0000","last_clean":"2024-04-25T01:01:38.415723+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:01:38.415723+0000","last_undegraded":"2024-04-25T01:01:38.415723+0000","last_fullsized":"2024-04-25T01:01:38.415723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416424+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:01:38.416424+0000","last_peered":"2024-04-25T01:01:38.416424+0000","last_clean":"2024-04-25T01:01:38.416424+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:01:38.416424+0000","last_undegraded":"2024-04-25T01:01:38.416424+0000","last_fullsized":"2024-04-25T01:01:38.416424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415424+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:01:38.415424+0000","last_peered":"2024-04-25T01:01:38.415424+0000","last_clean":"2024-04-25T01:01:38.415424+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:01:38.415424+0000","last_undegraded":"2024-04-25T01:01:38.415424+0000","last_fullsized":"2024-04-25T01:01:38.415424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097606+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:01:39.097606+0000","last_peered":"2024-04-25T01:01:39.097606+0000","last_clean":"2024-04-25T01:01:39.097606+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:01:39.097606+0000","last_undegraded":"2024-04-25T01:01:39.097606+0000","last_fullsized":"2024-04-25T01:01:39.097606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575834+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:01:39.575834+0000","last_peered":"2024-04-25T01:01:39.575834+0000","last_clean":"2024-04-25T01:01:39.575834+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:01:39.575834+0000","last_undegraded":"2024-04-25T01:01:39.575834+0000","last_fullsized":"2024-04-25T01:01:39.575834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459195+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:01:38.459195+0000","last_peered":"2024-04-25T01:01:38.459195+0000","last_clean":"2024-04-25T01:01:38.459195+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:01:38.459195+0000","last_undegraded":"2024-04-25T01:01:38.459195+0000","last_fullsized":"2024-04-25T01:01:38.459195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468951+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:01:39.468951+0000","last_peered":"2024-04-25T01:01:39.468951+0000","last_clean":"2024-04-25T01:01:39.468951+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:01:39.468951+0000","last_undegraded":"2024-04-25T01:01:39.468951+0000","last_fullsized":"2024-04-25T01:01:39.468951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415256+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:01:38.415256+0000","last_peered":"2024-04-25T01:01:38.415256+0000","last_clean":"2024-04-25T01:01:38.415256+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:01:38.415256+0000","last_undegraded":"2024-04-25T01:01:38.415256+0000","last_fullsized":"2024-04-25T01:01:38.415256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552244+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:01:38.552244+0000","last_peered":"2024-04-25T01:01:38.552244+0000","last_clean":"2024-04-25T01:01:38.552244+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:01:38.552244+0000","last_undegraded":"2024-04-25T01:01:38.552244+0000","last_fullsized":"2024-04-25T01:01:38.552244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415802+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:01:38.415802+0000","last_peered":"2024-04-25T01:01:38.415802+0000","last_clean":"2024-04-25T01:01:38.415802+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:01:38.415802+0000","last_undegraded":"2024-04-25T01:01:38.415802+0000","last_fullsized":"2024-04-25T01:01:38.415802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459070+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:01:38.459070+0000","last_peered":"2024-04-25T01:01:38.459070+0000","last_clean":"2024-04-25T01:01:38.459070+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:01:38.459070+0000","last_undegraded":"2024-04-25T01:01:38.459070+0000","last_fullsized":"2024-04-25T01:01:38.459070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415674+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:01:38.415674+0000","last_peered":"2024-04-25T01:01:38.415674+0000","last_clean":"2024-04-25T01:01:38.415674+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:01:38.415674+0000","last_undegraded":"2024-04-25T01:01:38.415674+0000","last_fullsized":"2024-04-25T01:01:38.415674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459211+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:01:38.459211+0000","last_peered":"2024-04-25T01:01:38.459211+0000","last_clean":"2024-04-25T01:01:38.459211+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:01:38.459211+0000","last_undegraded":"2024-04-25T01:01:38.459211+0000","last_fullsized":"2024-04-25T01:01:38.459211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468702+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:01:39.468702+0000","last_peered":"2024-04-25T01:01:39.468702+0000","last_clean":"2024-04-25T01:01:39.468702+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:01:39.468702+0000","last_undegraded":"2024-04-25T01:01:39.468702+0000","last_fullsized":"2024-04-25T01:01:39.468702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098040+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:01:39.098040+0000","last_peered":"2024-04-25T01:01:39.098040+0000","last_clean":"2024-04-25T01:01:39.098040+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:01:39.098040+0000","last_undegraded":"2024-04-25T01:01:39.098040+0000","last_fullsized":"2024-04-25T01:01:39.098040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417195+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:01:38.417195+0000","last_peered":"2024-04-25T01:01:38.417195+0000","last_clean":"2024-04-25T01:01:38.417195+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:01:38.417195+0000","last_undegraded":"2024-04-25T01:01:38.417195+0000","last_fullsized":"2024-04-25T01:01:38.417195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576085+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:01:39.576085+0000","last_peered":"2024-04-25T01:01:39.576085+0000","last_clean":"2024-04-25T01:01:39.576085+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:01:39.576085+0000","last_undegraded":"2024-04-25T01:01:39.576085+0000","last_fullsized":"2024-04-25T01:01:39.576085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459745+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:01:38.459745+0000","last_peered":"2024-04-25T01:01:38.459745+0000","last_clean":"2024-04-25T01:01:38.459745+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:01:38.459745+0000","last_undegraded":"2024-04-25T01:01:38.459745+0000","last_fullsized":"2024-04-25T01:01:38.459745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458917+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:01:38.458917+0000","last_peered":"2024-04-25T01:01:38.458917+0000","last_clean":"2024-04-25T01:01:38.458917+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:01:38.458917+0000","last_undegraded":"2024-04-25T01:01:38.458917+0000","last_fullsized":"2024-04-25T01:01:38.458917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551583+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:01:38.551583+0000","last_peered":"2024-04-25T01:01:38.551583+0000","last_clean":"2024-04-25T01:01:38.551583+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:01:38.551583+0000","last_undegraded":"2024-04-25T01:01:38.551583+0000","last_fullsized":"2024-04-25T01:01:38.551583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468688+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:01:39.468688+0000","last_peered":"2024-04-25T01:01:39.468688+0000","last_clean":"2024-04-25T01:01:39.468688+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:01:39.468688+0000","last_undegraded":"2024-04-25T01:01:39.468688+0000","last_fullsized":"2024-04-25T01:01:39.468688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415410+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:01:38.415410+0000","last_peered":"2024-04-25T01:01:38.415410+0000","last_clean":"2024-04-25T01:01:38.415410+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:01:38.415410+0000","last_undegraded":"2024-04-25T01:01:38.415410+0000","last_fullsized":"2024-04-25T01:01:38.415410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416131+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:01:38.416131+0000","last_peered":"2024-04-25T01:01:38.416131+0000","last_clean":"2024-04-25T01:01:38.416131+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:01:38.416131+0000","last_undegraded":"2024-04-25T01:01:38.416131+0000","last_fullsized":"2024-04-25T01:01:38.416131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098109+0000","last_change":"2024-04-25T01:01:33.405682+0000","last_active":"2024-04-25T01:01:39.098109+0000","last_peered":"2024-04-25T01:01:39.098109+0000","last_clean":"2024-04-25T01:01:39.098109+0000","last_became_active":"2024-04-25T01:01:33.405406+0000","last_became_peered":"2024-04-25T01:01:33.405406+0000","last_unstale":"2024-04-25T01:01:39.098109+0000","last_undegraded":"2024-04-25T01:01:39.098109+0000","last_fullsized":"2024-04-25T01:01:39.098109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098315+0000","last_change":"2024-04-25T01:01:33.414506+0000","last_active":"2024-04-25T01:01:39.098315+0000","last_peered":"2024-04-25T01:01:39.098315+0000","last_clean":"2024-04-25T01:01:39.098315+0000","last_became_active":"2024-04-25T01:01:33.414060+0000","last_became_peered":"2024-04-25T01:01:33.414060+0000","last_unstale":"2024-04-25T01:01:39.098315+0000","last_undegraded":"2024-04-25T01:01:39.098315+0000","last_fullsized":"2024-04-25T01:01:39.098315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417283+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:01:38.417283+0000","last_peered":"2024-04-25T01:01:38.417283+0000","last_clean":"2024-04-25T01:01:38.417283+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:01:38.417283+0000","last_undegraded":"2024-04-25T01:01:38.417283+0000","last_fullsized":"2024-04-25T01:01:38.417283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414797+0000","last_change":"2024-04-25T01:01:33.408326+0000","last_active":"2024-04-25T01:01:38.414797+0000","last_peered":"2024-04-25T01:01:38.414797+0000","last_clean":"2024-04-25T01:01:38.414797+0000","last_became_active":"2024-04-25T01:01:33.408039+0000","last_became_peered":"2024-04-25T01:01:33.408039+0000","last_unstale":"2024-04-25T01:01:38.414797+0000","last_undegraded":"2024-04-25T01:01:38.414797+0000","last_fullsized":"2024-04-25T01:01:38.414797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415697+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:01:38.415697+0000","last_peered":"2024-04-25T01:01:38.415697+0000","last_clean":"2024-04-25T01:01:38.415697+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:01:38.415697+0000","last_undegraded":"2024-04-25T01:01:38.415697+0000","last_fullsized":"2024-04-25T01:01:38.415697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459077+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:01:38.459077+0000","last_peered":"2024-04-25T01:01:38.459077+0000","last_clean":"2024-04-25T01:01:38.459077+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:01:38.459077+0000","last_undegraded":"2024-04-25T01:01:38.459077+0000","last_fullsized":"2024-04-25T01:01:38.459077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415902+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:01:38.415902+0000","last_peered":"2024-04-25T01:01:38.415902+0000","last_clean":"2024-04-25T01:01:38.415902+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:01:38.415902+0000","last_undegraded":"2024-04-25T01:01:38.415902+0000","last_fullsized":"2024-04-25T01:01:38.415902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459266+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:01:38.459266+0000","last_peered":"2024-04-25T01:01:38.459266+0000","last_clean":"2024-04-25T01:01:38.459266+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:01:38.459266+0000","last_undegraded":"2024-04-25T01:01:38.459266+0000","last_fullsized":"2024-04-25T01:01:38.459266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468558+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:01:39.468558+0000","last_peered":"2024-04-25T01:01:39.468558+0000","last_clean":"2024-04-25T01:01:39.468558+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:01:39.468558+0000","last_undegraded":"2024-04-25T01:01:39.468558+0000","last_fullsized":"2024-04-25T01:01:39.468558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552094+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:01:38.552094+0000","last_peered":"2024-04-25T01:01:38.552094+0000","last_clean":"2024-04-25T01:01:38.552094+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:01:38.552094+0000","last_undegraded":"2024-04-25T01:01:38.552094+0000","last_fullsized":"2024-04-25T01:01:38.552094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417169+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:01:38.417169+0000","last_peered":"2024-04-25T01:01:38.417169+0000","last_clean":"2024-04-25T01:01:38.417169+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:01:38.417169+0000","last_undegraded":"2024-04-25T01:01:38.417169+0000","last_fullsized":"2024-04-25T01:01:38.417169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:01:56.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:01:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.548+0000 7f0a4889e640 1 Processor -- start 2024-04-25T01:01:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.548+0000 7f0a4889e640 1 -- start start 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4889e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a4414f6f0 0x7f0a4414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4889e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a440c4f10 con 0x7f0a440abe50 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4889e640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a440c5180 con 0x7f0a440ac540 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4889e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0a44155720 con 0x7f0a4414f6f0 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4259c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a4414f6f0 0x7f0a4414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a42d9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0a440abe50 0x7f0a4406d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:57388/0 (socket says 172.21.15.129:57388) 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a42d9d640 1 -- 172.21.15.129:0/1960533575 learned_addr learned my addr 172.21.15.129:0/1960533575 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4259c640 1 -- 172.21.15.129:0/1960533575 >> v1:172.21.15.171:6789/0 conn(0x7f0a440ac540 legacy=0x7f0a440c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4259c640 1 -- 172.21.15.129:0/1960533575 >> v1:172.21.15.129:6789/0 conn(0x7f0a440abe50 legacy=0x7f0a4406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a4259c640 1 -- 172.21.15.129:0/1960533575 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a44153010 con 0x7f0a4414f6f0 2024-04-25T01:01:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.549+0000 7f0a41d9b640 1 -- 172.21.15.129:0/1960533575 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365677830 0 0) 0x7f0a440c4f10 con 0x7f0a440abe50 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4259c640 1 --2- 172.21.15.129:0/1960533575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a4414f6f0 0x7f0a4414fad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a340097a0 tx=0x7f0a3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1dd3ab43f20bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a41d9b640 1 -- 172.21.15.129:0/1960533575 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34037070 con 0x7f0a4414f6f0 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a41d9b640 1 -- 172.21.15.129:0/1960533575 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a34032440 con 0x7f0a4414f6f0 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a41d9b640 1 -- 172.21.15.129:0/1960533575 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34032810 con 0x7f0a4414f6f0 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 -- 172.21.15.129:0/1960533575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a4414f6f0 msgr2=0x7f0a4414fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 --2- 172.21.15.129:0/1960533575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a4414f6f0 0x7f0a4414fad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a340097a0 tx=0x7f0a3402a1a0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 -- 172.21.15.129:0/1960533575 shutdown_connections 2024-04-25T01:01:56.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 --2- 172.21.15.129:0/1960533575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a4414f6f0 0x7f0a4414fad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 -- 172.21.15.129:0/1960533575 >> 172.21.15.129:0/1960533575 conn(0x7f0a440a86b0 msgr2=0x7f0a440a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 -- 172.21.15.129:0/1960533575 shutdown_connections 2024-04-25T01:01:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.550+0000 7f0a4889e640 1 -- 172.21.15.129:0/1960533575 wait complete. 2024-04-25T01:01:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 Processor -- start 2024-04-25T01:01:56.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 -- start start 2024-04-25T01:01:56.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 0x7f0a441decb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a440ac540 0x7f0a4414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a4414f6f0 0x7f0a44149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44155fa0 con 0x7f0a440abe50 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0a44155cc0 con 0x7f0a4414f6f0 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4889e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4406ddd0 con 0x7f0a440ac540 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.552+0000 7f0a4259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 0x7f0a441decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 0x7f0a441decb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45496/0 (socket says 172.21.15.129:45496) 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 -- 172.21.15.129:0/2792580858 learned_addr learned my addr 172.21.15.129:0/2792580858 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a42d9d640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a4414f6f0 0x7f0a44149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a440ac540 0x7f0a4414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 -- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a440ac540 msgr2=0x7f0a4414e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a440ac540 0x7f0a4414e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 -- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a4414f6f0 msgr2=0x7f0a44149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a4414f6f0 0x7f0a44149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 -- 172.21.15.129:0/2792580858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a44155170 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a440ac540 0x7f0a4414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a42d9d640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a4414f6f0 0x7f0a44149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4259c640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 0x7f0a441decb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a340097a0 tx=0x7f0a340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34037050 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a34038040 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a44155450 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004490 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.553+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a441501d0 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.554+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a44075550 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.555+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a34030900 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.555+0000 7f0a3affd640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2406be30 0x7f0a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.555+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0a340abd90 con 0x7f0a440abe50 2024-04-25T01:01:56.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.555+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2406be30 0x7f0a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.556+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2406be30 0x7f0a2406e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c007fd0 tx=0x7f0a2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.559+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a3407a400 con 0x7f0a440abe50 2024-04-25T01:01:56.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.729+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a4406d7b0 con 0x7f0a2406be30 2024-04-25T01:01:56.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.737+0000 7f0a3affd640 1 -- 172.21.15.129:0/2792580858 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f0a4406d7b0 con 0x7f0a2406be30 2024-04-25T01:01:56.741 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:56.743 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:01:56.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2406be30 msgr2=0x7f0a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2406be30 0x7f0a2406e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c007fd0 tx=0x7f0a2c009040 comp rx=0 tx=0).stop 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 msgr2=0x7f0a441decb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 0x7f0a441decb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0a340097a0 tx=0x7f0a340046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 shutdown_connections 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2406be30 0x7f0a2406e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a440ac540 0x7f0a4414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a4414f6f0 0x7f0a44149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 --2- 172.21.15.129:0/2792580858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a440abe50 0x7f0a441decb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 >> 172.21.15.129:0/2792580858 conn(0x7f0a440a86b0 msgr2=0x7f0a440c1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:56.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 shutdown_connections 2024-04-25T01:01:56.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.741+0000 7f0a4889e640 1 -- 172.21.15.129:0/2792580858 wait complete. 2024-04-25T01:01:56.752 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T01:01:56.670825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219492,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755228,"statfs":{"total":773094113280,"available":772869353472,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003547"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468886+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:01:39.468886+0000","last_peered":"2024-04-25T01:01:39.468886+0000","last_clean":"2024-04-25T01:01:39.468886+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:01:39.468886+0000","last_undegraded":"2024-04-25T01:01:39.468886+0000","last_fullsized":"2024-04-25T01:01:39.468886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415373+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:01:38.415373+0000","last_peered":"2024-04-25T01:01:38.415373+0000","last_clean":"2024-04-25T01:01:38.415373+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:01:38.415373+0000","last_undegraded":"2024-04-25T01:01:38.415373+0000","last_fullsized":"2024-04-25T01:01:38.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414835+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:01:38.414835+0000","last_peered":"2024-04-25T01:01:38.414835+0000","last_clean":"2024-04-25T01:01:38.414835+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:01:38.414835+0000","last_undegraded":"2024-04-25T01:01:38.414835+0000","last_fullsized":"2024-04-25T01:01:38.414835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468718+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:01:39.468718+0000","last_peered":"2024-04-25T01:01:39.468718+0000","last_clean":"2024-04-25T01:01:39.468718+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:01:39.468718+0000","last_undegraded":"2024-04-25T01:01:39.468718+0000","last_fullsized":"2024-04-25T01:01:39.468718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415475+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:01:38.415475+0000","last_peered":"2024-04-25T01:01:38.415475+0000","last_clean":"2024-04-25T01:01:38.415475+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:01:38.415475+0000","last_undegraded":"2024-04-25T01:01:38.415475+0000","last_fullsized":"2024-04-25T01:01:38.415475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417116+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:01:38.417116+0000","last_peered":"2024-04-25T01:01:38.417116+0000","last_clean":"2024-04-25T01:01:38.417116+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:01:38.417116+0000","last_undegraded":"2024-04-25T01:01:38.417116+0000","last_fullsized":"2024-04-25T01:01:38.417116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459319+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:01:38.459319+0000","last_peered":"2024-04-25T01:01:38.459319+0000","last_clean":"2024-04-25T01:01:38.459319+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:01:38.459319+0000","last_undegraded":"2024-04-25T01:01:38.459319+0000","last_fullsized":"2024-04-25T01:01:38.459319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416961+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:01:38.416961+0000","last_peered":"2024-04-25T01:01:38.416961+0000","last_clean":"2024-04-25T01:01:38.416961+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:01:38.416961+0000","last_undegraded":"2024-04-25T01:01:38.416961+0000","last_fullsized":"2024-04-25T01:01:38.416961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416812+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:01:38.416812+0000","last_peered":"2024-04-25T01:01:38.416812+0000","last_clean":"2024-04-25T01:01:38.416812+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:01:38.416812+0000","last_undegraded":"2024-04-25T01:01:38.416812+0000","last_fullsized":"2024-04-25T01:01:38.416812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459435+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:01:38.459435+0000","last_peered":"2024-04-25T01:01:38.459435+0000","last_clean":"2024-04-25T01:01:38.459435+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:01:38.459435+0000","last_undegraded":"2024-04-25T01:01:38.459435+0000","last_fullsized":"2024-04-25T01:01:38.459435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416306+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:01:38.416306+0000","last_peered":"2024-04-25T01:01:38.416306+0000","last_clean":"2024-04-25T01:01:38.416306+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:01:38.416306+0000","last_undegraded":"2024-04-25T01:01:38.416306+0000","last_fullsized":"2024-04-25T01:01:38.416306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097726+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:01:39.097726+0000","last_peered":"2024-04-25T01:01:39.097726+0000","last_clean":"2024-04-25T01:01:39.097726+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:01:39.097726+0000","last_undegraded":"2024-04-25T01:01:39.097726+0000","last_fullsized":"2024-04-25T01:01:39.097726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416250+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:01:38.416250+0000","last_peered":"2024-04-25T01:01:38.416250+0000","last_clean":"2024-04-25T01:01:38.416250+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:01:38.416250+0000","last_undegraded":"2024-04-25T01:01:38.416250+0000","last_fullsized":"2024-04-25T01:01:38.416250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416610+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:01:38.416610+0000","last_peered":"2024-04-25T01:01:38.416610+0000","last_clean":"2024-04-25T01:01:38.416610+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:01:38.416610+0000","last_undegraded":"2024-04-25T01:01:38.416610+0000","last_fullsized":"2024-04-25T01:01:38.416610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415556+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:01:38.415556+0000","last_peered":"2024-04-25T01:01:38.415556+0000","last_clean":"2024-04-25T01:01:38.415556+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:01:38.415556+0000","last_undegraded":"2024-04-25T01:01:38.415556+0000","last_fullsized":"2024-04-25T01:01:38.415556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551986+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:01:38.551986+0000","last_peered":"2024-04-25T01:01:38.551986+0000","last_clean":"2024-04-25T01:01:38.551986+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:01:38.551986+0000","last_undegraded":"2024-04-25T01:01:38.551986+0000","last_fullsized":"2024-04-25T01:01:38.551986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415994+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:01:38.415994+0000","last_peered":"2024-04-25T01:01:38.415994+0000","last_clean":"2024-04-25T01:01:38.415994+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:01:38.415994+0000","last_undegraded":"2024-04-25T01:01:38.415994+0000","last_fullsized":"2024-04-25T01:01:38.415994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415087+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:01:38.415087+0000","last_peered":"2024-04-25T01:01:38.415087+0000","last_clean":"2024-04-25T01:01:38.415087+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:01:38.415087+0000","last_undegraded":"2024-04-25T01:01:38.415087+0000","last_fullsized":"2024-04-25T01:01:38.415087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551639+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:01:38.551639+0000","last_peered":"2024-04-25T01:01:38.551639+0000","last_clean":"2024-04-25T01:01:38.551639+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:01:38.551639+0000","last_undegraded":"2024-04-25T01:01:38.551639+0000","last_fullsized":"2024-04-25T01:01:38.551639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552136+0000","last_change":"2024-04-25T01:01:37.436247+0000","last_active":"2024-04-25T01:01:38.552136+0000","last_peered":"2024-04-25T01:01:38.552136+0000","last_clean":"2024-04-25T01:01:38.552136+0000","last_became_active":"2024-04-25T01:01:37.435388+0000","last_became_peered":"2024-04-25T01:01:37.435388+0000","last_unstale":"2024-04-25T01:01:38.552136+0000","last_undegraded":"2024-04-25T01:01:38.552136+0000","last_fullsized":"2024-04-25T01:01:38.552136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416152+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:01:38.416152+0000","last_peered":"2024-04-25T01:01:38.416152+0000","last_clean":"2024-04-25T01:01:38.416152+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:01:38.416152+0000","last_undegraded":"2024-04-25T01:01:38.416152+0000","last_fullsized":"2024-04-25T01:01:38.416152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415346+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:01:38.415346+0000","last_peered":"2024-04-25T01:01:38.415346+0000","last_clean":"2024-04-25T01:01:38.415346+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:01:38.415346+0000","last_undegraded":"2024-04-25T01:01:38.415346+0000","last_fullsized":"2024-04-25T01:01:38.415346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416144+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:01:38.416144+0000","last_peered":"2024-04-25T01:01:38.416144+0000","last_clean":"2024-04-25T01:01:38.416144+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:01:38.416144+0000","last_undegraded":"2024-04-25T01:01:38.416144+0000","last_fullsized":"2024-04-25T01:01:38.416144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468858+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:01:39.468858+0000","last_peered":"2024-04-25T01:01:39.468858+0000","last_clean":"2024-04-25T01:01:39.468858+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:01:39.468858+0000","last_undegraded":"2024-04-25T01:01:39.468858+0000","last_fullsized":"2024-04-25T01:01:39.468858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098463+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:01:39.098463+0000","last_peered":"2024-04-25T01:01:39.098463+0000","last_clean":"2024-04-25T01:01:39.098463+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:01:39.098463+0000","last_undegraded":"2024-04-25T01:01:39.098463+0000","last_fullsized":"2024-04-25T01:01:39.098463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415860+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:01:38.415860+0000","last_peered":"2024-04-25T01:01:38.415860+0000","last_clean":"2024-04-25T01:01:38.415860+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:01:38.415860+0000","last_undegraded":"2024-04-25T01:01:38.415860+0000","last_fullsized":"2024-04-25T01:01:38.415860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415706+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:01:38.415706+0000","last_peered":"2024-04-25T01:01:38.415706+0000","last_clean":"2024-04-25T01:01:38.415706+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:01:38.415706+0000","last_undegraded":"2024-04-25T01:01:38.415706+0000","last_fullsized":"2024-04-25T01:01:38.415706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552159+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:01:38.552159+0000","last_peered":"2024-04-25T01:01:38.552159+0000","last_clean":"2024-04-25T01:01:38.552159+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:01:38.552159+0000","last_undegraded":"2024-04-25T01:01:38.552159+0000","last_fullsized":"2024-04-25T01:01:38.552159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414731+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:01:38.414731+0000","last_peered":"2024-04-25T01:01:38.414731+0000","last_clean":"2024-04-25T01:01:38.414731+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:01:38.414731+0000","last_undegraded":"2024-04-25T01:01:38.414731+0000","last_fullsized":"2024-04-25T01:01:38.414731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552149+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:01:38.552149+0000","last_peered":"2024-04-25T01:01:38.552149+0000","last_clean":"2024-04-25T01:01:38.552149+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:01:38.552149+0000","last_undegraded":"2024-04-25T01:01:38.552149+0000","last_fullsized":"2024-04-25T01:01:38.552149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417245+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:01:38.417245+0000","last_peered":"2024-04-25T01:01:38.417245+0000","last_clean":"2024-04-25T01:01:38.417245+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:01:38.417245+0000","last_undegraded":"2024-04-25T01:01:38.417245+0000","last_fullsized":"2024-04-25T01:01:38.417245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468995+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:01:39.468995+0000","last_peered":"2024-04-25T01:01:39.468995+0000","last_clean":"2024-04-25T01:01:39.468995+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:01:39.468995+0000","last_undegraded":"2024-04-25T01:01:39.468995+0000","last_fullsized":"2024-04-25T01:01:39.468995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459566+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:01:38.459566+0000","last_peered":"2024-04-25T01:01:38.459566+0000","last_clean":"2024-04-25T01:01:38.459566+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:01:38.459566+0000","last_undegraded":"2024-04-25T01:01:38.459566+0000","last_fullsized":"2024-04-25T01:01:38.459566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552153+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:01:38.552153+0000","last_peered":"2024-04-25T01:01:38.552153+0000","last_clean":"2024-04-25T01:01:38.552153+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:01:38.552153+0000","last_undegraded":"2024-04-25T01:01:38.552153+0000","last_fullsized":"2024-04-25T01:01:38.552153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551709+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:01:38.551709+0000","last_peered":"2024-04-25T01:01:38.551709+0000","last_clean":"2024-04-25T01:01:38.551709+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:01:38.551709+0000","last_undegraded":"2024-04-25T01:01:38.551709+0000","last_fullsized":"2024-04-25T01:01:38.551709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416998+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:01:38.416998+0000","last_peered":"2024-04-25T01:01:38.416998+0000","last_clean":"2024-04-25T01:01:38.416998+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:01:38.416998+0000","last_undegraded":"2024-04-25T01:01:38.416998+0000","last_fullsized":"2024-04-25T01:01:38.416998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459358+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:01:38.459358+0000","last_peered":"2024-04-25T01:01:38.459358+0000","last_clean":"2024-04-25T01:01:38.459358+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:01:38.459358+0000","last_undegraded":"2024-04-25T01:01:38.459358+0000","last_fullsized":"2024-04-25T01:01:38.459358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552085+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:01:38.552085+0000","last_peered":"2024-04-25T01:01:38.552085+0000","last_clean":"2024-04-25T01:01:38.552085+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:01:38.552085+0000","last_undegraded":"2024-04-25T01:01:38.552085+0000","last_fullsized":"2024-04-25T01:01:38.552085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551641+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:01:38.551641+0000","last_peered":"2024-04-25T01:01:38.551641+0000","last_clean":"2024-04-25T01:01:38.551641+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:01:38.551641+0000","last_undegraded":"2024-04-25T01:01:38.551641+0000","last_fullsized":"2024-04-25T01:01:38.551641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575958+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:01:39.575958+0000","last_peered":"2024-04-25T01:01:39.575958+0000","last_clean":"2024-04-25T01:01:39.575958+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:01:39.575958+0000","last_undegraded":"2024-04-25T01:01:39.575958+0000","last_fullsized":"2024-04-25T01:01:39.575958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552170+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:01:38.552170+0000","last_peered":"2024-04-25T01:01:38.552170+0000","last_clean":"2024-04-25T01:01:38.552170+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:01:38.552170+0000","last_undegraded":"2024-04-25T01:01:38.552170+0000","last_fullsized":"2024-04-25T01:01:38.552170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468365+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:01:39.468365+0000","last_peered":"2024-04-25T01:01:39.468365+0000","last_clean":"2024-04-25T01:01:39.468365+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:01:39.468365+0000","last_undegraded":"2024-04-25T01:01:39.468365+0000","last_fullsized":"2024-04-25T01:01:39.468365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415600+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:01:38.415600+0000","last_peered":"2024-04-25T01:01:38.415600+0000","last_clean":"2024-04-25T01:01:38.415600+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:01:38.415600+0000","last_undegraded":"2024-04-25T01:01:38.415600+0000","last_fullsized":"2024-04-25T01:01:38.415600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468734+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:01:39.468734+0000","last_peered":"2024-04-25T01:01:39.468734+0000","last_clean":"2024-04-25T01:01:39.468734+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:01:39.468734+0000","last_undegraded":"2024-04-25T01:01:39.468734+0000","last_fullsized":"2024-04-25T01:01:39.468734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414950+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:01:38.414950+0000","last_peered":"2024-04-25T01:01:38.414950+0000","last_clean":"2024-04-25T01:01:38.414950+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:01:38.414950+0000","last_undegraded":"2024-04-25T01:01:38.414950+0000","last_fullsized":"2024-04-25T01:01:38.414950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459194+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:01:38.459194+0000","last_peered":"2024-04-25T01:01:38.459194+0000","last_clean":"2024-04-25T01:01:38.459194+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:01:38.459194+0000","last_undegraded":"2024-04-25T01:01:38.459194+0000","last_fullsized":"2024-04-25T01:01:38.459194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415719+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:01:38.415719+0000","last_peered":"2024-04-25T01:01:38.415719+0000","last_clean":"2024-04-25T01:01:38.415719+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:01:38.415719+0000","last_undegraded":"2024-04-25T01:01:38.415719+0000","last_fullsized":"2024-04-25T01:01:38.415719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098131+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:01:39.098131+0000","last_peered":"2024-04-25T01:01:39.098131+0000","last_clean":"2024-04-25T01:01:39.098131+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:01:39.098131+0000","last_undegraded":"2024-04-25T01:01:39.098131+0000","last_fullsized":"2024-04-25T01:01:39.098131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417023+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:01:38.417023+0000","last_peered":"2024-04-25T01:01:38.417023+0000","last_clean":"2024-04-25T01:01:38.417023+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:01:38.417023+0000","last_undegraded":"2024-04-25T01:01:38.417023+0000","last_fullsized":"2024-04-25T01:01:38.417023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576180+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:01:39.576180+0000","last_peered":"2024-04-25T01:01:39.576180+0000","last_clean":"2024-04-25T01:01:39.576180+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:01:39.576180+0000","last_undegraded":"2024-04-25T01:01:39.576180+0000","last_fullsized":"2024-04-25T01:01:39.576180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097822+0000","last_change":"2024-04-25T01:01:37.432081+0000","last_active":"2024-04-25T01:01:39.097822+0000","last_peered":"2024-04-25T01:01:39.097822+0000","last_clean":"2024-04-25T01:01:39.097822+0000","last_became_active":"2024-04-25T01:01:37.431657+0000","last_became_peered":"2024-04-25T01:01:37.431657+0000","last_unstale":"2024-04-25T01:01:39.097822+0000","last_undegraded":"2024-04-25T01:01:39.097822+0000","last_fullsized":"2024-04-25T01:01:39.097822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459452+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:01:38.459452+0000","last_peered":"2024-04-25T01:01:38.459452+0000","last_clean":"2024-04-25T01:01:38.459452+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:01:38.459452+0000","last_undegraded":"2024-04-25T01:01:38.459452+0000","last_fullsized":"2024-04-25T01:01:38.459452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552164+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:01:38.552164+0000","last_peered":"2024-04-25T01:01:38.552164+0000","last_clean":"2024-04-25T01:01:38.552164+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:01:38.552164+0000","last_undegraded":"2024-04-25T01:01:38.552164+0000","last_fullsized":"2024-04-25T01:01:38.552164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468921+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:01:39.468921+0000","last_peered":"2024-04-25T01:01:39.468921+0000","last_clean":"2024-04-25T01:01:39.468921+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:01:39.468921+0000","last_undegraded":"2024-04-25T01:01:39.468921+0000","last_fullsized":"2024-04-25T01:01:39.468921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417383+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:01:38.417383+0000","last_peered":"2024-04-25T01:01:38.417383+0000","last_clean":"2024-04-25T01:01:38.417383+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:01:38.417383+0000","last_undegraded":"2024-04-25T01:01:38.417383+0000","last_fullsized":"2024-04-25T01:01:38.417383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576094+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:01:39.576094+0000","last_peered":"2024-04-25T01:01:39.576094+0000","last_clean":"2024-04-25T01:01:39.576094+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:01:39.576094+0000","last_undegraded":"2024-04-25T01:01:39.576094+0000","last_fullsized":"2024-04-25T01:01:39.576094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458996+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:01:38.458996+0000","last_peered":"2024-04-25T01:01:38.458996+0000","last_clean":"2024-04-25T01:01:38.458996+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:01:38.458996+0000","last_undegraded":"2024-04-25T01:01:38.458996+0000","last_fullsized":"2024-04-25T01:01:38.458996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416863+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:01:38.416863+0000","last_peered":"2024-04-25T01:01:38.416863+0000","last_clean":"2024-04-25T01:01:38.416863+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:01:38.416863+0000","last_undegraded":"2024-04-25T01:01:38.416863+0000","last_fullsized":"2024-04-25T01:01:38.416863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416428+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:01:38.416428+0000","last_peered":"2024-04-25T01:01:38.416428+0000","last_clean":"2024-04-25T01:01:38.416428+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:01:38.416428+0000","last_undegraded":"2024-04-25T01:01:38.416428+0000","last_fullsized":"2024-04-25T01:01:38.416428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415619+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:01:38.415619+0000","last_peered":"2024-04-25T01:01:38.415619+0000","last_clean":"2024-04-25T01:01:38.415619+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:01:38.415619+0000","last_undegraded":"2024-04-25T01:01:38.415619+0000","last_fullsized":"2024-04-25T01:01:38.415619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415805+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:01:38.415805+0000","last_peered":"2024-04-25T01:01:38.415805+0000","last_clean":"2024-04-25T01:01:38.415805+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:01:38.415805+0000","last_undegraded":"2024-04-25T01:01:38.415805+0000","last_fullsized":"2024-04-25T01:01:38.415805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575808+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:01:39.575808+0000","last_peered":"2024-04-25T01:01:39.575808+0000","last_clean":"2024-04-25T01:01:39.575808+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:01:39.575808+0000","last_undegraded":"2024-04-25T01:01:39.575808+0000","last_fullsized":"2024-04-25T01:01:39.575808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468765+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:01:39.468765+0000","last_peered":"2024-04-25T01:01:39.468765+0000","last_clean":"2024-04-25T01:01:39.468765+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:01:39.468765+0000","last_undegraded":"2024-04-25T01:01:39.468765+0000","last_fullsized":"2024-04-25T01:01:39.468765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097771+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:01:39.097771+0000","last_peered":"2024-04-25T01:01:39.097771+0000","last_clean":"2024-04-25T01:01:39.097771+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:01:39.097771+0000","last_undegraded":"2024-04-25T01:01:39.097771+0000","last_fullsized":"2024-04-25T01:01:39.097771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097883+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:01:39.097883+0000","last_peered":"2024-04-25T01:01:39.097883+0000","last_clean":"2024-04-25T01:01:39.097883+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:01:39.097883+0000","last_undegraded":"2024-04-25T01:01:39.097883+0000","last_fullsized":"2024-04-25T01:01:39.097883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415652+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:01:38.415652+0000","last_peered":"2024-04-25T01:01:38.415652+0000","last_clean":"2024-04-25T01:01:38.415652+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:01:38.415652+0000","last_undegraded":"2024-04-25T01:01:38.415652+0000","last_fullsized":"2024-04-25T01:01:38.415652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416147+0000","last_change":"2024-04-25T01:01:33.389402+0000","last_active":"2024-04-25T01:01:38.416147+0000","last_peered":"2024-04-25T01:01:38.416147+0000","last_clean":"2024-04-25T01:01:38.416147+0000","last_became_active":"2024-04-25T01:01:33.389147+0000","last_became_peered":"2024-04-25T01:01:33.389147+0000","last_unstale":"2024-04-25T01:01:38.416147+0000","last_undegraded":"2024-04-25T01:01:38.416147+0000","last_fullsized":"2024-04-25T01:01:38.416147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459311+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:01:38.459311+0000","last_peered":"2024-04-25T01:01:38.459311+0000","last_clean":"2024-04-25T01:01:38.459311+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:01:38.459311+0000","last_undegraded":"2024-04-25T01:01:38.459311+0000","last_fullsized":"2024-04-25T01:01:38.459311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415085+0000","last_change":"2024-04-25T01:01:37.441477+0000","last_active":"2024-04-25T01:01:38.415085+0000","last_peered":"2024-04-25T01:01:38.415085+0000","last_clean":"2024-04-25T01:01:38.415085+0000","last_became_active":"2024-04-25T01:01:37.441099+0000","last_became_peered":"2024-04-25T01:01:37.441099+0000","last_unstale":"2024-04-25T01:01:38.415085+0000","last_undegraded":"2024-04-25T01:01:38.415085+0000","last_fullsized":"2024-04-25T01:01:38.415085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468846+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:01:39.468846+0000","last_peered":"2024-04-25T01:01:39.468846+0000","last_clean":"2024-04-25T01:01:39.468846+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:01:39.468846+0000","last_undegraded":"2024-04-25T01:01:39.468846+0000","last_fullsized":"2024-04-25T01:01:39.468846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469030+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:01:39.469030+0000","last_peered":"2024-04-25T01:01:39.469030+0000","last_clean":"2024-04-25T01:01:39.469030+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:01:39.469030+0000","last_undegraded":"2024-04-25T01:01:39.469030+0000","last_fullsized":"2024-04-25T01:01:39.469030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459574+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:01:38.459574+0000","last_peered":"2024-04-25T01:01:38.459574+0000","last_clean":"2024-04-25T01:01:38.459574+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:01:38.459574+0000","last_undegraded":"2024-04-25T01:01:38.459574+0000","last_fullsized":"2024-04-25T01:01:38.459574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416957+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:01:38.416957+0000","last_peered":"2024-04-25T01:01:38.416957+0000","last_clean":"2024-04-25T01:01:38.416957+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:01:38.416957+0000","last_undegraded":"2024-04-25T01:01:38.416957+0000","last_fullsized":"2024-04-25T01:01:38.416957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:43.083494+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:01:43.083494+0000","last_peered":"2024-04-25T01:01:43.083494+0000","last_clean":"2024-04-25T01:01:43.083494+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:01:43.083494+0000","last_undegraded":"2024-04-25T01:01:43.083494+0000","last_fullsized":"2024-04-25T01:01:43.083494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551940+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:01:38.551940+0000","last_peered":"2024-04-25T01:01:38.551940+0000","last_clean":"2024-04-25T01:01:38.551940+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:01:38.551940+0000","last_undegraded":"2024-04-25T01:01:38.551940+0000","last_fullsized":"2024-04-25T01:01:38.551940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416242+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:01:38.416242+0000","last_peered":"2024-04-25T01:01:38.416242+0000","last_clean":"2024-04-25T01:01:38.416242+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:01:38.416242+0000","last_undegraded":"2024-04-25T01:01:38.416242+0000","last_fullsized":"2024-04-25T01:01:38.416242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552062+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:01:38.552062+0000","last_peered":"2024-04-25T01:01:38.552062+0000","last_clean":"2024-04-25T01:01:38.552062+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:01:38.552062+0000","last_undegraded":"2024-04-25T01:01:38.552062+0000","last_fullsized":"2024-04-25T01:01:38.552062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416785+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:01:38.416785+0000","last_peered":"2024-04-25T01:01:38.416785+0000","last_clean":"2024-04-25T01:01:38.416785+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:01:38.416785+0000","last_undegraded":"2024-04-25T01:01:38.416785+0000","last_fullsized":"2024-04-25T01:01:38.416785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468828+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:01:39.468828+0000","last_peered":"2024-04-25T01:01:39.468828+0000","last_clean":"2024-04-25T01:01:39.468828+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:01:39.468828+0000","last_undegraded":"2024-04-25T01:01:39.468828+0000","last_fullsized":"2024-04-25T01:01:39.468828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459534+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:01:38.459534+0000","last_peered":"2024-04-25T01:01:38.459534+0000","last_clean":"2024-04-25T01:01:38.459534+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:01:38.459534+0000","last_undegraded":"2024-04-25T01:01:38.459534+0000","last_fullsized":"2024-04-25T01:01:38.459534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459526+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:01:38.459526+0000","last_peered":"2024-04-25T01:01:38.459526+0000","last_clean":"2024-04-25T01:01:38.459526+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:01:38.459526+0000","last_undegraded":"2024-04-25T01:01:38.459526+0000","last_fullsized":"2024-04-25T01:01:38.459526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414973+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:01:38.414973+0000","last_peered":"2024-04-25T01:01:38.414973+0000","last_clean":"2024-04-25T01:01:38.414973+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:01:38.414973+0000","last_undegraded":"2024-04-25T01:01:38.414973+0000","last_fullsized":"2024-04-25T01:01:38.414973+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468563+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:01:39.468563+0000","last_peered":"2024-04-25T01:01:39.468563+0000","last_clean":"2024-04-25T01:01:39.468563+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:01:39.468563+0000","last_undegraded":"2024-04-25T01:01:39.468563+0000","last_fullsized":"2024-04-25T01:01:39.468563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551523+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:01:38.551523+0000","last_peered":"2024-04-25T01:01:38.551523+0000","last_clean":"2024-04-25T01:01:38.551523+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:01:38.551523+0000","last_undegraded":"2024-04-25T01:01:38.551523+0000","last_fullsized":"2024-04-25T01:01:38.551523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459092+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:01:38.459092+0000","last_peered":"2024-04-25T01:01:38.459092+0000","last_clean":"2024-04-25T01:01:38.459092+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:01:38.459092+0000","last_undegraded":"2024-04-25T01:01:38.459092+0000","last_fullsized":"2024-04-25T01:01:38.459092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098352+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:01:39.098352+0000","last_peered":"2024-04-25T01:01:39.098352+0000","last_clean":"2024-04-25T01:01:39.098352+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:01:39.098352+0000","last_undegraded":"2024-04-25T01:01:39.098352+0000","last_fullsized":"2024-04-25T01:01:39.098352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414947+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:01:38.414947+0000","last_peered":"2024-04-25T01:01:38.414947+0000","last_clean":"2024-04-25T01:01:38.414947+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:01:38.414947+0000","last_undegraded":"2024-04-25T01:01:38.414947+0000","last_fullsized":"2024-04-25T01:01:38.414947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415843+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:01:38.415843+0000","last_peered":"2024-04-25T01:01:38.415843+0000","last_clean":"2024-04-25T01:01:38.415843+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:01:38.415843+0000","last_undegraded":"2024-04-25T01:01:38.415843+0000","last_fullsized":"2024-04-25T01:01:38.415843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097915+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:01:39.097915+0000","last_peered":"2024-04-25T01:01:39.097915+0000","last_clean":"2024-04-25T01:01:39.097915+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:01:39.097915+0000","last_undegraded":"2024-04-25T01:01:39.097915+0000","last_fullsized":"2024-04-25T01:01:39.097915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469001+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:01:39.469001+0000","last_peered":"2024-04-25T01:01:39.469001+0000","last_clean":"2024-04-25T01:01:39.469001+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:01:39.469001+0000","last_undegraded":"2024-04-25T01:01:39.469001+0000","last_fullsized":"2024-04-25T01:01:39.469001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576127+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:01:39.576127+0000","last_peered":"2024-04-25T01:01:39.576127+0000","last_clean":"2024-04-25T01:01:39.576127+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:01:39.576127+0000","last_undegraded":"2024-04-25T01:01:39.576127+0000","last_fullsized":"2024-04-25T01:01:39.576127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415723+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:01:38.415723+0000","last_peered":"2024-04-25T01:01:38.415723+0000","last_clean":"2024-04-25T01:01:38.415723+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:01:38.415723+0000","last_undegraded":"2024-04-25T01:01:38.415723+0000","last_fullsized":"2024-04-25T01:01:38.415723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416424+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:01:38.416424+0000","last_peered":"2024-04-25T01:01:38.416424+0000","last_clean":"2024-04-25T01:01:38.416424+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:01:38.416424+0000","last_undegraded":"2024-04-25T01:01:38.416424+0000","last_fullsized":"2024-04-25T01:01:38.416424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415424+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:01:38.415424+0000","last_peered":"2024-04-25T01:01:38.415424+0000","last_clean":"2024-04-25T01:01:38.415424+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:01:38.415424+0000","last_undegraded":"2024-04-25T01:01:38.415424+0000","last_fullsized":"2024-04-25T01:01:38.415424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097606+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:01:39.097606+0000","last_peered":"2024-04-25T01:01:39.097606+0000","last_clean":"2024-04-25T01:01:39.097606+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:01:39.097606+0000","last_undegraded":"2024-04-25T01:01:39.097606+0000","last_fullsized":"2024-04-25T01:01:39.097606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575834+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:01:39.575834+0000","last_peered":"2024-04-25T01:01:39.575834+0000","last_clean":"2024-04-25T01:01:39.575834+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:01:39.575834+0000","last_undegraded":"2024-04-25T01:01:39.575834+0000","last_fullsized":"2024-04-25T01:01:39.575834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459195+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:01:38.459195+0000","last_peered":"2024-04-25T01:01:38.459195+0000","last_clean":"2024-04-25T01:01:38.459195+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:01:38.459195+0000","last_undegraded":"2024-04-25T01:01:38.459195+0000","last_fullsized":"2024-04-25T01:01:38.459195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468951+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:01:39.468951+0000","last_peered":"2024-04-25T01:01:39.468951+0000","last_clean":"2024-04-25T01:01:39.468951+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:01:39.468951+0000","last_undegraded":"2024-04-25T01:01:39.468951+0000","last_fullsized":"2024-04-25T01:01:39.468951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415256+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:01:38.415256+0000","last_peered":"2024-04-25T01:01:38.415256+0000","last_clean":"2024-04-25T01:01:38.415256+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:01:38.415256+0000","last_undegraded":"2024-04-25T01:01:38.415256+0000","last_fullsized":"2024-04-25T01:01:38.415256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552244+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:01:38.552244+0000","last_peered":"2024-04-25T01:01:38.552244+0000","last_clean":"2024-04-25T01:01:38.552244+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:01:38.552244+0000","last_undegraded":"2024-04-25T01:01:38.552244+0000","last_fullsized":"2024-04-25T01:01:38.552244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415802+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:01:38.415802+0000","last_peered":"2024-04-25T01:01:38.415802+0000","last_clean":"2024-04-25T01:01:38.415802+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:01:38.415802+0000","last_undegraded":"2024-04-25T01:01:38.415802+0000","last_fullsized":"2024-04-25T01:01:38.415802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459070+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:01:38.459070+0000","last_peered":"2024-04-25T01:01:38.459070+0000","last_clean":"2024-04-25T01:01:38.459070+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:01:38.459070+0000","last_undegraded":"2024-04-25T01:01:38.459070+0000","last_fullsized":"2024-04-25T01:01:38.459070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415674+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:01:38.415674+0000","last_peered":"2024-04-25T01:01:38.415674+0000","last_clean":"2024-04-25T01:01:38.415674+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:01:38.415674+0000","last_undegraded":"2024-04-25T01:01:38.415674+0000","last_fullsized":"2024-04-25T01:01:38.415674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459211+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:01:38.459211+0000","last_peered":"2024-04-25T01:01:38.459211+0000","last_clean":"2024-04-25T01:01:38.459211+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:01:38.459211+0000","last_undegraded":"2024-04-25T01:01:38.459211+0000","last_fullsized":"2024-04-25T01:01:38.459211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468702+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:01:39.468702+0000","last_peered":"2024-04-25T01:01:39.468702+0000","last_clean":"2024-04-25T01:01:39.468702+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:01:39.468702+0000","last_undegraded":"2024-04-25T01:01:39.468702+0000","last_fullsized":"2024-04-25T01:01:39.468702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098040+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:01:39.098040+0000","last_peered":"2024-04-25T01:01:39.098040+0000","last_clean":"2024-04-25T01:01:39.098040+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:01:39.098040+0000","last_undegraded":"2024-04-25T01:01:39.098040+0000","last_fullsized":"2024-04-25T01:01:39.098040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417195+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:01:38.417195+0000","last_peered":"2024-04-25T01:01:38.417195+0000","last_clean":"2024-04-25T01:01:38.417195+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:01:38.417195+0000","last_undegraded":"2024-04-25T01:01:38.417195+0000","last_fullsized":"2024-04-25T01:01:38.417195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576085+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:01:39.576085+0000","last_peered":"2024-04-25T01:01:39.576085+0000","last_clean":"2024-04-25T01:01:39.576085+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:01:39.576085+0000","last_undegraded":"2024-04-25T01:01:39.576085+0000","last_fullsized":"2024-04-25T01:01:39.576085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459745+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:01:38.459745+0000","last_peered":"2024-04-25T01:01:38.459745+0000","last_clean":"2024-04-25T01:01:38.459745+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:01:38.459745+0000","last_undegraded":"2024-04-25T01:01:38.459745+0000","last_fullsized":"2024-04-25T01:01:38.459745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458917+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:01:38.458917+0000","last_peered":"2024-04-25T01:01:38.458917+0000","last_clean":"2024-04-25T01:01:38.458917+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:01:38.458917+0000","last_undegraded":"2024-04-25T01:01:38.458917+0000","last_fullsized":"2024-04-25T01:01:38.458917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551583+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:01:38.551583+0000","last_peered":"2024-04-25T01:01:38.551583+0000","last_clean":"2024-04-25T01:01:38.551583+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:01:38.551583+0000","last_undegraded":"2024-04-25T01:01:38.551583+0000","last_fullsized":"2024-04-25T01:01:38.551583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468688+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:01:39.468688+0000","last_peered":"2024-04-25T01:01:39.468688+0000","last_clean":"2024-04-25T01:01:39.468688+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:01:39.468688+0000","last_undegraded":"2024-04-25T01:01:39.468688+0000","last_fullsized":"2024-04-25T01:01:39.468688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415410+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:01:38.415410+0000","last_peered":"2024-04-25T01:01:38.415410+0000","last_clean":"2024-04-25T01:01:38.415410+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:01:38.415410+0000","last_undegraded":"2024-04-25T01:01:38.415410+0000","last_fullsized":"2024-04-25T01:01:38.415410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416131+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:01:38.416131+0000","last_peered":"2024-04-25T01:01:38.416131+0000","last_clean":"2024-04-25T01:01:38.416131+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:01:38.416131+0000","last_undegraded":"2024-04-25T01:01:38.416131+0000","last_fullsized":"2024-04-25T01:01:38.416131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098109+0000","last_change":"2024-04-25T01:01:33.405682+0000","last_active":"2024-04-25T01:01:39.098109+0000","last_peered":"2024-04-25T01:01:39.098109+0000","last_clean":"2024-04-25T01:01:39.098109+0000","last_became_active":"2024-04-25T01:01:33.405406+0000","last_became_peered":"2024-04-25T01:01:33.405406+0000","last_unstale":"2024-04-25T01:01:39.098109+0000","last_undegraded":"2024-04-25T01:01:39.098109+0000","last_fullsized":"2024-04-25T01:01:39.098109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098315+0000","last_change":"2024-04-25T01:01:33.414506+0000","last_active":"2024-04-25T01:01:39.098315+0000","last_peered":"2024-04-25T01:01:39.098315+0000","last_clean":"2024-04-25T01:01:39.098315+0000","last_became_active":"2024-04-25T01:01:33.414060+0000","last_became_peered":"2024-04-25T01:01:33.414060+0000","last_unstale":"2024-04-25T01:01:39.098315+0000","last_undegraded":"2024-04-25T01:01:39.098315+0000","last_fullsized":"2024-04-25T01:01:39.098315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417283+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:01:38.417283+0000","last_peered":"2024-04-25T01:01:38.417283+0000","last_clean":"2024-04-25T01:01:38.417283+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:01:38.417283+0000","last_undegraded":"2024-04-25T01:01:38.417283+0000","last_fullsized":"2024-04-25T01:01:38.417283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414797+0000","last_change":"2024-04-25T01:01:33.408326+0000","last_active":"2024-04-25T01:01:38.414797+0000","last_peered":"2024-04-25T01:01:38.414797+0000","last_clean":"2024-04-25T01:01:38.414797+0000","last_became_active":"2024-04-25T01:01:33.408039+0000","last_became_peered":"2024-04-25T01:01:33.408039+0000","last_unstale":"2024-04-25T01:01:38.414797+0000","last_undegraded":"2024-04-25T01:01:38.414797+0000","last_fullsized":"2024-04-25T01:01:38.414797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415697+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:01:38.415697+0000","last_peered":"2024-04-25T01:01:38.415697+0000","last_clean":"2024-04-25T01:01:38.415697+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:01:38.415697+0000","last_undegraded":"2024-04-25T01:01:38.415697+0000","last_fullsized":"2024-04-25T01:01:38.415697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459077+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:01:38.459077+0000","last_peered":"2024-04-25T01:01:38.459077+0000","last_clean":"2024-04-25T01:01:38.459077+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:01:38.459077+0000","last_undegraded":"2024-04-25T01:01:38.459077+0000","last_fullsized":"2024-04-25T01:01:38.459077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415902+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:01:38.415902+0000","last_peered":"2024-04-25T01:01:38.415902+0000","last_clean":"2024-04-25T01:01:38.415902+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:01:38.415902+0000","last_undegraded":"2024-04-25T01:01:38.415902+0000","last_fullsized":"2024-04-25T01:01:38.415902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459266+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:01:38.459266+0000","last_peered":"2024-04-25T01:01:38.459266+0000","last_clean":"2024-04-25T01:01:38.459266+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:01:38.459266+0000","last_undegraded":"2024-04-25T01:01:38.459266+0000","last_fullsized":"2024-04-25T01:01:38.459266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468558+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:01:39.468558+0000","last_peered":"2024-04-25T01:01:39.468558+0000","last_clean":"2024-04-25T01:01:39.468558+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:01:39.468558+0000","last_undegraded":"2024-04-25T01:01:39.468558+0000","last_fullsized":"2024-04-25T01:01:39.468558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552094+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:01:38.552094+0000","last_peered":"2024-04-25T01:01:38.552094+0000","last_clean":"2024-04-25T01:01:38.552094+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:01:38.552094+0000","last_undegraded":"2024-04-25T01:01:38.552094+0000","last_fullsized":"2024-04-25T01:01:38.552094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417169+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:01:38.417169+0000","last_peered":"2024-04-25T01:01:38.417169+0000","last_clean":"2024-04-25T01:01:38.417169+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:01:38.417169+0000","last_undegraded":"2024-04-25T01:01:38.417169+0000","last_fullsized":"2024-04-25T01:01:38.417169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:01:56.757 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:01:56.757 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T01:01:56.758 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T01:01:56.758 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T01:01:56.758 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T01:01:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.889+0000 7f36c07ce640 1 Processor -- start 2024-04-25T01:01:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.889+0000 7f36c07ce640 1 -- start start 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36c07ce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 0x7f36b80c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36c07ce640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f36b806cb10 0x7f36b806d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36c07ce640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b806e780 con 0x7f36b80c0470 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36c07ce640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f36b806c680 con 0x7f36b806cb10 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36c07ce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f36b80c0970 con 0x7f36b8152bc0 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 0x7f36b80c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 0x7f36b80c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:51818/0 (socket says 172.21.15.129:51818) 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 -- 172.21.15.129:0/2208265892 learned_addr learned my addr 172.21.15.129:0/2208265892 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36bed6b640 1 --2- 172.21.15.129:0/2208265892 >> v2:172.21.15.171:3300/0 conn(0x7f36b806cb10 0x7f36b806d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 -- 172.21.15.129:0/2208265892 >> v2:172.21.15.171:3300/0 conn(0x7f36b806cb10 msgr2=0x7f36b806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:56.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 --2- 172.21.15.129:0/2208265892 >> v2:172.21.15.171:3300/0 conn(0x7f36b806cb10 0x7f36b806d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 -- 172.21.15.129:0/2208265892 >> v1:172.21.15.171:6789/0 conn(0x7f36b80c0470 legacy=0x7f36b8159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36be56a640 1 -- 172.21.15.129:0/2208265892 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b80c0c50 con 0x7f36b8152bc0 2024-04-25T01:01:56.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.890+0000 7f36bed6b640 1 --2- 172.21.15.129:0/2208265892 >> v2:172.21.15.171:3300/0 conn(0x7f36b806cb10 0x7f36b806d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.891+0000 7f36be56a640 1 --2- 172.21.15.129:0/2208265892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 0x7f36b80c20d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009870 tx=0x7f36b402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03da9bd7fb1cb3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.891+0000 7f36bd568640 1 -- 172.21.15.129:0/2208265892 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4037070 con 0x7f36b8152bc0 2024-04-25T01:01:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.891+0000 7f36bd568640 1 -- 172.21.15.129:0/2208265892 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b4032440 con 0x7f36b8152bc0 2024-04-25T01:01:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.891+0000 7f36bd568640 1 -- 172.21.15.129:0/2208265892 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4032810 con 0x7f36b8152bc0 2024-04-25T01:01:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 -- 172.21.15.129:0/2208265892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 msgr2=0x7f36b80c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2208265892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 0x7f36b80c20d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009870 tx=0x7f36b402a270 comp rx=0 tx=0).stop 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 -- 172.21.15.129:0/2208265892 shutdown_connections 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2208265892 >> v2:172.21.15.171:3300/0 conn(0x7f36b806cb10 0x7f36b806d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2208265892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b8152bc0 0x7f36b80c20d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 -- 172.21.15.129:0/2208265892 >> 172.21.15.129:0/2208265892 conn(0x7f36b80a86b0 msgr2=0x7f36b80a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 -- 172.21.15.129:0/2208265892 shutdown_connections 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.892+0000 7f36c07ce640 1 -- 172.21.15.129:0/2208265892 wait complete. 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.893+0000 7f36c07ce640 1 Processor -- start 2024-04-25T01:01:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.893+0000 7f36c07ce640 1 -- start start 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b806cb10 0x7f36b81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 0x7f36b81ee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36b8152bc0 0x7f36b814aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36b806cfc0 con 0x7f36b80c0470 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80c0f30 con 0x7f36b8152bc0 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f36b806d140 con 0x7f36b806cb10 2024-04-25T01:01:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 0x7f36b81ee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 0x7f36b81ee5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45498/0 (socket says 172.21.15.129:45498) 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 -- 172.21.15.129:0/2478320524 learned_addr learned my addr 172.21.15.129:0/2478320524 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bed6b640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36b8152bc0 0x7f36b814aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36be56a640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b806cb10 0x7f36b81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 -- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b806cb10 msgr2=0x7f36b81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b806cb10 0x7f36b81531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 -- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36b8152bc0 msgr2=0x7f36b814aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36b8152bc0 0x7f36b814aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 -- 172.21.15.129:0/2478320524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b8155fe0 con 0x7f36b80c0470 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36bdd69640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 0x7f36b81ee5e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c440 tx=0x7f36a8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a80052d0 con 0x7f36b80c0470 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a8005470 con 0x7f36b80c0470 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8007260 con 0x7f36b80c0470 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b81ef060 con 0x7f36b80c0470 2024-04-25T01:01:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.894+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b8073450 con 0x7f36b80c0470 2024-04-25T01:01:56.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.895+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b8075550 con 0x7f36b80c0470 2024-04-25T01:01:56.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.896+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36a8007400 con 0x7f36b80c0470 2024-04-25T01:01:56.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.896+0000 7f36aeffd640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f369c06be30 0x7f369c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:56.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.896+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f36a808a480 con 0x7f36b80c0470 2024-04-25T01:01:56.898 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.896+0000 7f36be56a640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f369c06be30 0x7f369c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:56.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.897+0000 7f36be56a640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f369c06be30 0x7f369c06e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36b4034000 tx=0x7f36b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:56.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:56.900+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f36a8058530 con 0x7f36b80c0470 2024-04-25T01:01:57.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.123+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f36b806dd50 con 0x7f36b80c0470 2024-04-25T01:01:57.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.126+0000 7f36aeffd640 1 -- 172.21.15.129:0/2478320524 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172945 (secure 0 0 0) 0x7f36a805c830 con 0x7f36b80c0470 2024-04-25T01:01:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:57.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f369c06be30 msgr2=0x7f369c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f369c06be30 0x7f369c06e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36b4034000 tx=0x7f36b402e000 comp rx=0 tx=0).stop 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 msgr2=0x7f36b81ee5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 0x7f36b81ee5e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c440 tx=0x7f36a8001e70 comp rx=0 tx=0).stop 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 shutdown_connections 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f369c06be30 0x7f369c06e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36b8152bc0 0x7f36b814aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36b80c0470 0x7f36b81ee5e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 --2- 172.21.15.129:0/2478320524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36b806cb10 0x7f36b81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 >> 172.21.15.129:0/2478320524 conn(0x7f36b80a86b0 msgr2=0x7f36b8157060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 shutdown_connections 2024-04-25T01:01:57.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.129+0000 7f36c07ce640 1 -- 172.21.15.129:0/2478320524 wait complete. 2024-04-25T01:01:57.140 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1668644098},{"type":"v1","addr":"172.21.15.171:6833","nonce":1668644098}]},"active_addr":"172.21.15.171:6833/1668644098","active_change":"2024-04-25T01:01:12.650052+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":395216354}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":3084937459}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":2190110845}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":3574627462}]}]} 2024-04-25T01:01:57.143 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T01:01:57.143 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T01:01:57.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.239+0000 7fcea3329640 1 Processor -- start 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.239+0000 7fcea3329640 1 -- start start 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea3329640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fce9c0a7e20 0x7fce9c159450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea3329640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea3329640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce9c0a84e0 con 0x7fce9c156170 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea3329640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c159990 con 0x7fce9c0a7e20 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea3329640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c06d330 con 0x7fce9c0c6e90 2024-04-25T01:01:57.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea10c5640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fce9c156170 0x7fce9c0aa6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:57394/0 (socket says 172.21.15.129:57394) 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea10c5640 1 -- 172.21.15.129:0/3371831268 learned_addr learned my addr 172.21.15.129:0/3371831268 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea18c6640 1 --2- 172.21.15.129:0/3371831268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fcea08c4640 1 --2- 172.21.15.129:0/3371831268 >> v2:172.21.15.171:3300/0 conn(0x7fce9c0a7e20 0x7fce9c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.240+0000 7fce93fff640 1 -- 172.21.15.129:0/3371831268 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 919526117 0 0) 0x7fce9c0a84e0 con 0x7fce9c156170 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.241+0000 7fce93fff640 1 -- 172.21.15.129:0/3371831268 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce88003520 con 0x7fce9c156170 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.241+0000 7fcea18c6640 1 -- 172.21.15.129:0/3371831268 >> v2:172.21.15.171:3300/0 conn(0x7fce9c0a7e20 msgr2=0x7fce9c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.241+0000 7fcea18c6640 1 --2- 172.21.15.129:0/3371831268 >> v2:172.21.15.171:3300/0 conn(0x7fce9c0a7e20 0x7fce9c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.241+0000 7fcea18c6640 1 -- 172.21.15.129:0/3371831268 >> v1:172.21.15.129:6789/0 conn(0x7fce9c156170 legacy=0x7fce9c0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.241+0000 7fcea18c6640 1 -- 172.21.15.129:0/3371831268 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c06ca00 con 0x7fce9c0c6e90 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.241+0000 7fcea18c6640 1 --2- 172.21.15.129:0/3371831268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c0c7300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce94009d90 tx=0x7fce9402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2087c164e31a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fce93fff640 1 -- 172.21.15.129:0/3371831268 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94007740 con 0x7fce9c0c6e90 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fce93fff640 1 -- 172.21.15.129:0/3371831268 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce940078e0 con 0x7fce9c0c6e90 2024-04-25T01:01:57.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fce93fff640 1 -- 172.21.15.129:0/3371831268 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94007c90 con 0x7fce9c0c6e90 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 -- 172.21.15.129:0/3371831268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 msgr2=0x7fce9c0c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 --2- 172.21.15.129:0/3371831268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c0c7300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce94009d90 tx=0x7fce9402a660 comp rx=0 tx=0).stop 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 -- 172.21.15.129:0/3371831268 shutdown_connections 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 --2- 172.21.15.129:0/3371831268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c0c7300 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 --2- 172.21.15.129:0/3371831268 >> v2:172.21.15.171:3300/0 conn(0x7fce9c0a7e20 0x7fce9c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 -- 172.21.15.129:0/3371831268 >> 172.21.15.129:0/3371831268 conn(0x7fce9c0c5490 msgr2=0x7fce9c0c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 -- 172.21.15.129:0/3371831268 shutdown_connections 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.242+0000 7fcea3329640 1 -- 172.21.15.129:0/3371831268 wait complete. 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.243+0000 7fcea3329640 1 Processor -- start 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.243+0000 7fcea3329640 1 -- start start 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea3329640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 0x7fce9c1ee940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea3329640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea3329640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce9c156170 0x7fce9c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea3329640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c06cfc0 con 0x7fce9c0a7e20 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea3329640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0a8280 con 0x7fce9c156170 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea3329640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0aade0 con 0x7fce9c0c6e90 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 0x7fce9c1ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 0x7fce9c1ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45506/0 (socket says 172.21.15.129:45506) 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 -- 172.21.15.129:0/237931526 learned_addr learned my addr 172.21.15.129:0/237931526 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea08c4640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea18c6640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce9c156170 0x7fce9c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 -- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 msgr2=0x7fce9c150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 -- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce9c156170 msgr2=0x7fce9c150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce9c156170 0x7fce9c150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea10c5640 1 -- 172.21.15.129:0/237931526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c1df270 con 0x7fce9c0a7e20 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea08c4640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:57.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.244+0000 7fcea18c6640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce9c156170 0x7fce9c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.245+0000 7fcea10c5640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 0x7fce9c1ee940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c390 tx=0x7fce9800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.245+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98017070 con 0x7fce9c0a7e20 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.245+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce98005ce0 con 0x7fce9c0a7e20 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.245+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c150f30 con 0x7fce9c0a7e20 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.245+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce980072f0 con 0x7fce9c0a7e20 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.245+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c070880 con 0x7fce9c0a7e20 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.246+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c075550 con 0x7fce9c0a7e20 2024-04-25T01:01:57.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.246+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce9801a070 con 0x7fce9c0a7e20 2024-04-25T01:01:57.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.247+0000 7fce91ffb640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.247+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce9808a550 con 0x7fce9c0a7e20 2024-04-25T01:01:57.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.247+0000 7fcea08c4640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.247+0000 7fcea08c4640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fce8006be30 0x7fce8006e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce9c1e8fa0 tx=0x7fce8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.251+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fce98058c50 con 0x7fce9c0a7e20 2024-04-25T01:01:57.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.428+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce9c0c6c70 con 0x7fce9c0a7e20 2024-04-25T01:01:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.429+0000 7fce91ffb640 1 -- 172.21.15.129:0/237931526 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fce98058550 con 0x7fce9c0a7e20 2024-04-25T01:01:57.430 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:57.430 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fce8006be30 msgr2=0x7fce8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fce8006be30 0x7fce8006e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce9c1e8fa0 tx=0x7fce8c008040 comp rx=0 tx=0).stop 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 msgr2=0x7fce9c1ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 0x7fce9c1ee940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c390 tx=0x7fce9800c850 comp rx=0 tx=0).stop 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 shutdown_connections 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fce8006be30 0x7fce8006e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fce9c0c6e90 0x7fce9c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce9c156170 0x7fce9c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 --2- 172.21.15.129:0/237931526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce9c0a7e20 0x7fce9c1ee940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 >> 172.21.15.129:0/237931526 conn(0x7fce9c0c5490 msgr2=0x7fce9c0c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 shutdown_connections 2024-04-25T01:01:57.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.431+0000 7fcea3329640 1 -- 172.21.15.129:0/237931526 wait complete. 2024-04-25T01:01:57.443 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T01:01:57.443 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:01:57.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.540+0000 7fa397d22640 1 Processor -- start 2024-04-25T01:01:57.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.540+0000 7fa397d22640 1 -- start start 2024-04-25T01:01:57.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa397d22640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa39014f260 0x7fa3900c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa397d22640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa390155d70 0x7fa3900c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa397d22640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3900733a0 con 0x7fa390067300 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa397d22640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fa390156180 con 0x7fa390155d70 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa397d22640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fa390158c50 con 0x7fa39014f260 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa390155d70 0x7fa3900c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa390155d70 0x7fa3900c1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38376/0 (socket says 172.21.15.129:38376) 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 -- 172.21.15.129:0/3334301669 learned_addr learned my addr 172.21.15.129:0/3334301669 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa395abe640 1 --2- 172.21.15.129:0/3334301669 >> v2:172.21.15.171:3300/0 conn(0x7fa39014f260 0x7fa3900c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 -- 172.21.15.129:0/3334301669 >> v2:172.21.15.171:3300/0 conn(0x7fa39014f260 msgr2=0x7fa3900c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 --2- 172.21.15.129:0/3334301669 >> v2:172.21.15.171:3300/0 conn(0x7fa39014f260 0x7fa3900c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 -- 172.21.15.129:0/3334301669 >> v1:172.21.15.171:6789/0 conn(0x7fa390067300 legacy=0x7fa3901590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.541+0000 7fa3962bf640 1 -- 172.21.15.129:0/3334301669 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa390073680 con 0x7fa390155d70 2024-04-25T01:01:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.542+0000 7fa3962bf640 1 --2- 172.21.15.129:0/3334301669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c1bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa39006f040 tx=0x7fa38802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f81f625bca3c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.542+0000 7fa394abc640 1 -- 172.21.15.129:0/3334301669 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388037070 con 0x7fa390155d70 2024-04-25T01:01:57.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.542+0000 7fa394abc640 1 -- 172.21.15.129:0/3334301669 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa388032440 con 0x7fa390155d70 2024-04-25T01:01:57.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.542+0000 7fa394abc640 1 -- 172.21.15.129:0/3334301669 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388032810 con 0x7fa390155d70 2024-04-25T01:01:57.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 -- 172.21.15.129:0/3334301669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 msgr2=0x7fa3900c1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 --2- 172.21.15.129:0/3334301669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c1bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa39006f040 tx=0x7fa38802a1a0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 -- 172.21.15.129:0/3334301669 shutdown_connections 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 --2- 172.21.15.129:0/3334301669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c1bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 --2- 172.21.15.129:0/3334301669 >> v2:172.21.15.171:3300/0 conn(0x7fa39014f260 0x7fa3900c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 -- 172.21.15.129:0/3334301669 >> 172.21.15.129:0/3334301669 conn(0x7fa3900c71f0 msgr2=0x7fa3900c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 -- 172.21.15.129:0/3334301669 shutdown_connections 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.543+0000 7fa397d22640 1 -- 172.21.15.129:0/3334301669 wait complete. 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.544+0000 7fa397d22640 1 Processor -- start 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.544+0000 7fa397d22640 1 -- start start 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa390067300 0x7fa390153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39014f260 0x7fa3901582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa39006b400 con 0x7fa390155d70 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901597f0 con 0x7fa39014f260 2024-04-25T01:01:57.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa39014f640 con 0x7fa390067300 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38380/0 (socket says 172.21.15.129:38380) 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 -- 172.21.15.129:0/3459477581 learned_addr learned my addr 172.21.15.129:0/3459477581 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa395abe640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa390067300 0x7fa390153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3952bd640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39014f260 0x7fa3901582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 -- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa390067300 msgr2=0x7fa390153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa390067300 0x7fa390153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 -- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39014f260 msgr2=0x7fa3901582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39014f260 0x7fa3901582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 -- 172.21.15.129:0/3459477581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa390072f20 con 0x7fa390155d70 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa395abe640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa390067300 0x7fa390153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3952bd640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39014f260 0x7fa3901582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3962bf640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c62b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa3880090b0 tx=0x7fa3880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388041020 con 0x7fa390155d70 2024-04-25T01:01:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3880306c0 con 0x7fa390155d70 2024-04-25T01:01:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa390154e30 con 0x7fa390155d70 2024-04-25T01:01:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388037040 con 0x7fa390155d70 2024-04-25T01:01:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.545+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3900c67f0 con 0x7fa390155d70 2024-04-25T01:01:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.546+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa390075550 con 0x7fa390155d70 2024-04-25T01:01:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.547+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa388044020 con 0x7fa390155d70 2024-04-25T01:01:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.547+0000 7fa3867fc640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa37406be30 0x7fa37406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.547+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3880ab910 con 0x7fa390155d70 2024-04-25T01:01:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.547+0000 7fa395abe640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa37406be30 0x7fa37406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.548+0000 7fa395abe640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa37406be30 0x7fa37406e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007330 tx=0x7fa38c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.551+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa38804b030 con 0x7fa390155d70 2024-04-25T01:01:57.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.725+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa390069060 con 0x7fa390155d70 2024-04-25T01:01:57.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.726+0000 7fa3867fc640 1 -- 172.21.15.129:0/3459477581 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fa38807a010 con 0x7fa390155d70 2024-04-25T01:01:57.727 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:01:57.727 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa37406be30 msgr2=0x7fa37406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa37406be30 0x7fa37406e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007330 tx=0x7fa38c00b040 comp rx=0 tx=0).stop 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 msgr2=0x7fa3900c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c62b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa3880090b0 tx=0x7fa3880046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 shutdown_connections 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa37406be30 0x7fa37406e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa390155d70 0x7fa3900c62b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39014f260 0x7fa3901582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 --2- 172.21.15.129:0/3459477581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa390067300 0x7fa390153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 >> 172.21.15.129:0/3459477581 conn(0x7fa3900c71f0 msgr2=0x7fa39014c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 shutdown_connections 2024-04-25T01:01:57.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.728+0000 7fa397d22640 1 -- 172.21.15.129:0/3459477581 wait complete. 2024-04-25T01:01:57.740 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T01:01:57.741 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T01:01:57.741 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T01:01:57.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T01:01:57.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T01:01:57.742 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T01:01:57.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T01:01:57.743 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T01:01:57.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.909+0000 7f1bd4f1d640 1 Processor -- start 2024-04-25T01:01:57.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.910+0000 7fd9bf3cf640 1 Processor -- start 2024-04-25T01:01:57.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.910+0000 7fbaaf9e9640 1 Processor -- start 2024-04-25T01:01:57.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.909+0000 7f616ce00640 1 Processor -- start 2024-04-25T01:01:57.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.909+0000 7f616ce00640 1 -- start start 2024-04-25T01:01:57.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.910+0000 7fd9bf3cf640 1 -- start start 2024-04-25T01:01:57.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f616ce00640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f61600071a0 0x7f6160007580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.910+0000 7f1bd4f1d640 1 -- start start 2024-04-25T01:01:57.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f616ce00640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600e2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f616ce00640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616004a550 con 0x7f616000e9b0 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f616ce00640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f616000e520 con 0x7f61600071a0 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f616ce00640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f61600e2c60 con 0x7f616004db40 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f6165d9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f6165d9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600e2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47540/0 (socket says 172.21.15.129:47540) 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f6165d9b640 1 -- 172.21.15.129:0/2429968926 learned_addr learned my addr 172.21.15.129:0/2429968926 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f616659c640 1 --2- 172.21.15.129:0/2429968926 >> v2:172.21.15.129:3300/0 conn(0x7f61600071a0 0x7f6160007580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f616659c640 1 -- 172.21.15.129:0/2429968926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 msgr2=0x7f61600e2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bd4f1d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 0x7f1bd00c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bd4f1d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd006bf80 con 0x7f1bd00c3da0 2024-04-25T01:01:57.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bd4f1d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd00c7330 con 0x7f1bd00a71d0 2024-04-25T01:01:57.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bd4f1d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd014d8e0 con 0x7f1bd006f320 2024-04-25T01:01:57.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bced9d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 0x7f1bd00c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f367e929640 1 Processor -- start 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7f367e929640 1 -- start start 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f367e929640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f36781561f0 0x7f36780c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.910+0000 7fbaaf9e9640 1 -- start start 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaaf9e9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbaa8156cb0 0x7fbaa80c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaaf9e9640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaaf9e9640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa80c16f0 con 0x7fbaa8159790 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaaf9e9640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa8157090 con 0x7fbaa8156cb0 2024-04-25T01:01:57.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaaf9e9640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa806e270 con 0x7fbaa80ab410 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd3760cd640 1 Processor -- start 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f616659c640 1 --2- 172.21.15.129:0/2429968926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600e2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f616659c640 1 -- 172.21.15.129:0/2429968926 >> v1:172.21.15.129:6789/0 conn(0x7f616000e9b0 legacy=0x7f61600de790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f616659c640 1 -- 172.21.15.129:0/2429968926 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616000e1f0 con 0x7f61600071a0 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bcdd9b640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f1bd00a71d0 0x7f1bd00c5830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54028/0 (socket says 172.21.15.129:54028) 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bced9d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 0x7f1bd00c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47564/0 (socket says 172.21.15.129:47564) 2024-04-25T01:01:57.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f1bced9d640 1 -- 172.21.15.129:0/2938356752 learned_addr learned my addr 172.21.15.129:0/2938356752 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f367e929640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36780c0280 con 0x7f367806dd80 2024-04-25T01:01:57.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f367e929640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3678159990 con 0x7f3678155e10 2024-04-25T01:01:57.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f367e929640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f3678159470 con 0x7f36781561f0 2024-04-25T01:01:57.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f367cec6640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f367806dd80 0x7f36780c2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44662/0 (socket says 172.21.15.129:44662) 2024-04-25T01:01:57.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f367cec6640 1 -- 172.21.15.129:0/1426524795 learned_addr learned my addr 172.21.15.129:0/1426524795 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:57.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fa8095ab640 1 Processor -- start 2024-04-25T01:01:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaad785640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbaa8156cb0 0x7fbaa80c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f041ffff640 1 Processor -- start 2024-04-25T01:01:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaad785640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbaa8156cb0 0x7fbaa80c11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38402/0 (socket says 172.21.15.129:38402) 2024-04-25T01:01:57.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaacf84640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fa8095ab640 1 -- start start 2024-04-25T01:01:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd3760cd640 1 -- start start 2024-04-25T01:01:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd9bf3cf640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd9b80c6ec0 0x7fd9b80c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fd3760cd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd3680dbcd0 0x7fd368047490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd9bf3cf640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd9b806e4b0 0x7fd9b8163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd9bf3cf640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9b815a590 con 0x7fd9b806e0d0 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd9bf3cf640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b806e960 con 0x7fd9b806e4b0 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.911+0000 7fd9bf3cf640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fd9b815a100 con 0x7fd9b80c6ec0 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fd9be3cd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fd9b806e0d0 0x7fd9b80c5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44630/0 (socket says 172.21.15.129:44630) 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fd9be3cd640 1 -- 172.21.15.129:0/4252443951 learned_addr learned my addr 172.21.15.129:0/4252443951 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:57.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7f041ffff640 1 -- start start 2024-04-25T01:01:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fd9bebce640 1 --2- 172.21.15.129:0/4252443951 >> v2:172.21.15.129:3300/0 conn(0x7fd9b806e4b0 0x7fd9b8163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fd3760cd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd3680df170 0x7fd368006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bdbcc640 1 --2- 172.21.15.129:0/4252443951 >> v2:172.21.15.171:3300/0 conn(0x7fd9b80c6ec0 0x7fd9b80c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd3760cd640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36800bb40 con 0x7fd3680073d0 2024-04-25T01:01:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277642553 0 0) 0x7f1bd00c7330 con 0x7f1bd00a71d0 2024-04-25T01:01:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bb8003520 con 0x7f1bd00a71d0 2024-04-25T01:01:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd3760cd640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd36800a410 con 0x7fd3680dbcd0 2024-04-25T01:01:57.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd3760cd640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fd368048050 con 0x7fd3680df170 2024-04-25T01:01:57.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bd3cb640 1 -- 172.21.15.129:0/4252443951 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034909294 0 0) 0x7fd9b815a590 con 0x7fd9b806e0d0 2024-04-25T01:01:57.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bd3cb640 1 -- 172.21.15.129:0/4252443951 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a4003520 con 0x7fd9b806e0d0 2024-04-25T01:01:57.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bdbcc640 1 -- 172.21.15.129:0/4252443951 >> v2:172.21.15.129:3300/0 conn(0x7fd9b806e4b0 msgr2=0x7fd9b8163f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 <== mon.1 v1:172.21.15.171:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1052806936 0 0) 0x7f1bb8003520 con 0x7f1bd00a71d0 2024-04-25T01:01:57.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 msgr2=0x7f1bd00c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 0x7f1bd00c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 >> v1:172.21.15.129:6789/0 conn(0x7f1bd00c3da0 legacy=0x7f1bd006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 --> v1:172.21.15.171:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd00c7050 con 0x7f1bd00a71d0 2024-04-25T01:01:57.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd3680df170 0x7fd368006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd3680df170 0x7fd368006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54428/0 (socket says 172.21.15.129:54428) 2024-04-25T01:01:57.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 -- 172.21.15.129:0/691101888 learned_addr learned my addr 172.21.15.129:0/691101888 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36f7fe640 1 --2- 172.21.15.129:0/691101888 >> v2:172.21.15.129:3300/0 conn(0x7fd3680dbcd0 0x7fd368047490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 <== mon.1 v1:172.21.15.171:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f1bc4001670 con 0x7f1bd00a71d0 2024-04-25T01:01:57.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 >> v1:172.21.15.171:6789/0 conn(0x7f1bd00a71d0 legacy=0x7f1bd00c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bdbcc640 1 --2- 172.21.15.129:0/4252443951 >> v2:172.21.15.129:3300/0 conn(0x7fd9b806e4b0 0x7fd9b8163f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.912+0000 7fbaad785640 1 -- 172.21.15.129:0/1411580419 learned_addr learned my addr 172.21.15.129:0/1411580419 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fba9bfff640 1 -- 172.21.15.129:0/1411580419 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4198327973 0 0) 0x7fbaa80c16f0 con 0x7fbaa8159790 2024-04-25T01:01:57.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 -- 172.21.15.129:0/691101888 >> v2:172.21.15.129:3300/0 conn(0x7fd3680dbcd0 msgr2=0x7fd368047490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 --2- 172.21.15.129:0/691101888 >> v2:172.21.15.129:3300/0 conn(0x7fd3680dbcd0 0x7fd368047490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 -- 172.21.15.129:0/691101888 >> v1:172.21.15.129:6789/0 conn(0x7fd3680073d0 legacy=0x7fd36800a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb80039d0 0x7f1bb8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 0x7f1bb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bb8004d40 0x7f1bb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd014d8e0 con 0x7f1bb80039d0 2024-04-25T01:01:57.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb800fe80 con 0x7f1bb8004d40 2024-04-25T01:01:57.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb8010030 con 0x7f1bd006f320 2024-04-25T01:01:57.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bdbcc640 1 -- 172.21.15.129:0/4252443951 >> v1:172.21.15.129:6789/0 conn(0x7fd9b806e0d0 legacy=0x7fd9b80c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd9bdbcc640 1 -- 172.21.15.129:0/4252443951 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b806f340 con 0x7fd9b80c6ec0 2024-04-25T01:01:57.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fa8095ab640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 0x7fa7fc03ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f616659c640 1 --2- 172.21.15.129:0/2429968926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61600071a0 0x7f6160007580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f615c0097a0 tx=0x7f615c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8e4fbc31f353ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f36777fe640 1 --2- 172.21.15.129:0/1426524795 >> v2:172.21.15.129:3300/0 conn(0x7f36781561f0 0x7f36780c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263015869 0 0) 0x7f3678159990 con 0x7f3678155e10 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3668003520 con 0x7f3678155e10 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041ffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f041800a8b0 0x7f041800ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fa8095ab640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa7fc03f160 0x7fa7fc047cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fa8095ab640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc00bb30 con 0x7fa7fc004b30 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fa8095ab640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc006fc0 con 0x7fa7fc03f160 2024-04-25T01:01:57.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fa8095ab640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc007560 con 0x7fa7fc0dc020 2024-04-25T01:01:57.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.913+0000 7fd36effd640 1 -- 172.21.15.129:0/691101888 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36800a0e0 con 0x7fd3680df170 2024-04-25T01:01:57.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bce59c640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb80039d0 0x7f1bb8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fa802ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 0x7fa7fc03ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fa802ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 0x7fa7fc03ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47594/0 (socket says 172.21.15.129:47594) 2024-04-25T01:01:57.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fa802ffd640 1 -- 172.21.15.129:0/2618330871 learned_addr learned my addr 172.21.15.129:0/2618330871 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fbaacf84640 1 -- 172.21.15.129:0/1411580419 >> v2:172.21.15.129:3300/0 conn(0x7fbaa8156cb0 msgr2=0x7fbaa80c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fbaacf84640 1 --2- 172.21.15.129:0/1411580419 >> v2:172.21.15.129:3300/0 conn(0x7fbaa8156cb0 0x7fbaa80c11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fbaacf84640 1 -- 172.21.15.129:0/1411580419 >> v1:172.21.15.129:6789/0 conn(0x7fbaa8159790 legacy=0x7fbaa8159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fbaacf84640 1 -- 172.21.15.129:0/1411580419 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa806df90 con 0x7fbaa80ab410 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bced9d640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bb8004d40 0x7f1bb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bce59c640 1 -- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 msgr2=0x7f1bb8004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bce59c640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd006f320 0x7f1bb8004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bce59c640 1 -- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bb8004d40 msgr2=0x7f1bb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bce59c640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bb8004d40 0x7f1bb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bce59c640 1 -- 172.21.15.129:0/2938356752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd00c7050 con 0x7f1bb80039d0 2024-04-25T01:01:57.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f1bced9d640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bb8004d40 0x7f1bb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:57.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fa8037fe640 1 --2- 172.21.15.129:0/2618330871 >> v2:172.21.15.171:3300/0 conn(0x7fa7fc03f160 0x7fa7fc047cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7fd36effd640 1 --2- 172.21.15.129:0/691101888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680df170 0x7fd368006e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd36800be00 tx=0x7fd35c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f04a478f781ae698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980889745 0 0) 0x7f36780c0280 con 0x7f367806dd80 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3678159990 con 0x7f367806dd80 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.1 v1:172.21.15.171:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1862004200 0 0) 0x7f3668003520 con 0x7f3678155e10 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 >> v2:172.21.15.129:3300/0 conn(0x7f36781561f0 msgr2=0x7f36780c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f3676ffd640 1 --2- 172.21.15.129:0/1426524795 >> v2:172.21.15.129:3300/0 conn(0x7f36781561f0 0x7f36780c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 >> v1:172.21.15.129:6789/0 conn(0x7f367806dd80 legacy=0x7f36780c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 --> v1:172.21.15.171:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36780c6110 con 0x7f3678155e10 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.1 v1:172.21.15.171:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f36700019a0 con 0x7f3678155e10 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 >> v1:172.21.15.171:6789/0 conn(0x7f3678155e10 legacy=0x7f36780bfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fd9bdbcc640 1 --2- 172.21.15.129:0/4252443951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 0x7fd9b80c5e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9b806b190 tx=0x7fd9b402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e1018382d47665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3668003b40 0x7f3668003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fd36e7fc640 1 -- 172.21.15.129:0/691101888 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c037070 con 0x7fd3680df170 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3668005210 0x7f366800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3678159470 con 0x7f3668004800 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f366800c160 con 0x7f3668005210 2024-04-25T01:01:57.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f366800c360 con 0x7f3668003b40 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fa801ffb640 1 -- 172.21.15.129:0/2618330871 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011524082 0 0) 0x7fa7fc00bb30 con 0x7fa7fc004b30 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fa8037fe640 1 -- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 msgr2=0x7fa7fc03ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fa8037fe640 1 --2- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 0x7fa7fc03ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fa8037fe640 1 -- 172.21.15.129:0/2618330871 >> v1:172.21.15.171:6789/0 conn(0x7fa7fc004b30 legacy=0x7fa7fc0475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fa8037fe640 1 -- 172.21.15.129:0/2618330871 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc00be10 con 0x7fa7fc03f160 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fa802ffd640 1 --2- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 0x7fa7fc03ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:57.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fd36e7fc640 1 -- 172.21.15.129:0/691101888 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c032440 con 0x7fd3680df170 2024-04-25T01:01:57.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fd36e7fc640 1 -- 172.21.15.129:0/691101888 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c032850 con 0x7fd3680df170 2024-04-25T01:01:57.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f3677fff640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3668003b40 0x7f3668003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f616559a640 1 -- 172.21.15.129:0/2429968926 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c037070 con 0x7f61600071a0 2024-04-25T01:01:57.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f616559a640 1 -- 172.21.15.129:0/2429968926 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c032440 con 0x7f61600071a0 2024-04-25T01:01:57.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f616559a640 1 -- 172.21.15.129:0/2429968926 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c032810 con 0x7f61600071a0 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f367cec6640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3668005210 0x7f366800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fbaacf84640 1 --2- 172.21.15.129:0/1411580419 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80ab870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40097a0 tx=0x7fbaa402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ded9d47de61db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041ffff640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041800bb50 con 0x7f04180474a0 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041ffff640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0418048060 con 0x7f04180dbce0 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041ffff640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f041800a420 con 0x7f041800a8b0 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041e7fc640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f04180474a0 0x7f0418006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44700/0 (socket says 172.21.15.129:44700) 2024-04-25T01:01:57.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041e7fc640 1 -- 172.21.15.129:0/151033856 learned_addr learned my addr 172.21.15.129:0/151033856 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041f7fe640 1 --2- 172.21.15.129:0/151033856 >> v2:172.21.15.129:3300/0 conn(0x7f041800a8b0 0x7f041800ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041f7fe640 1 -- 172.21.15.129:0/151033856 >> v1:172.21.15.171:6789/0 conn(0x7f04180dbce0 legacy=0x7f04180df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041f7fe640 1 -- 172.21.15.129:0/151033856 >> v1:172.21.15.129:6789/0 conn(0x7f04180474a0 legacy=0x7f0418006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.914+0000 7f041f7fe640 1 -- 172.21.15.129:0/151033856 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041800a0f0 con 0x7f041800a8b0 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7f36777fe640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f1bce59c640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb80039d0 0x7f1bb8003e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80044d0 tx=0x7f1bb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f937c1e192584174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fba9bfff640 1 -- 172.21.15.129:0/1411580419 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4037070 con 0x7fbaa80ab410 2024-04-25T01:01:57.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fba9bfff640 1 -- 172.21.15.129:0/1411580419 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa402adb0 con 0x7fbaa80ab410 2024-04-25T01:01:57.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fba9bfff640 1 -- 172.21.15.129:0/1411580419 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4032660 con 0x7fbaa80ab410 2024-04-25T01:01:57.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f041f7fe640 1 --2- 172.21.15.129:0/151033856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041800a8b0 0x7f041800ad00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04200502b0 tx=0x7f04200a5a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da285917b5447934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fd9bd3cb640 1 -- 172.21.15.129:0/4252443951 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4037070 con 0x7fd9b80c6ec0 2024-04-25T01:01:57.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.915+0000 7fd9bd3cb640 1 -- 172.21.15.129:0/4252443951 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b4032440 con 0x7fd9b80c6ec0 2024-04-25T01:01:57.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bb80039d0 2024-04-25T01:01:57.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc802ad80 con 0x7f1bb80039d0 2024-04-25T01:01:57.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 -- 172.21.15.129:0/691101888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680df170 msgr2=0x7fd368006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 --2- 172.21.15.129:0/691101888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680df170 0x7fd368006e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd36800be00 tx=0x7fd35c02a1c0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f041dffb640 1 -- 172.21.15.129:0/151033856 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200ae070 con 0x7f041800a8b0 2024-04-25T01:01:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 -- 172.21.15.129:0/691101888 shutdown_connections 2024-04-25T01:01:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 --2- 172.21.15.129:0/691101888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680df170 0x7fd368006e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 --2- 172.21.15.129:0/691101888 >> v2:172.21.15.129:3300/0 conn(0x7fd3680dbcd0 0x7fd368047490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 -- 172.21.15.129:0/691101888 >> 172.21.15.129:0/691101888 conn(0x7fd368048240 msgr2=0x7fd368048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f041dffb640 1 -- 172.21.15.129:0/151033856 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04200aa930 con 0x7f041800a8b0 2024-04-25T01:01:57.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f041dffb640 1 -- 172.21.15.129:0/151033856 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200aada0 con 0x7f041800a8b0 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f36777fe640 1 -- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3668003b40 msgr2=0x7f3668003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f36777fe640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3668003b40 0x7f3668003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f36777fe640 1 -- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3668005210 msgr2=0x7f366800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f36777fe640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3668005210 0x7f366800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f36777fe640 1 -- 172.21.15.129:0/1426524795 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3668003520 con 0x7f3668004800 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f367cec6640 1 -- 172.21.15.129:0/1426524795 reap_dead start 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f616ce00640 1 -- 172.21.15.129:0/2429968926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61600071a0 msgr2=0x7f6160007580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7f616ce00640 1 --2- 172.21.15.129:0/2429968926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61600071a0 0x7f6160007580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f615c0097a0 tx=0x7f615c02a180 comp rx=0 tx=0).stop 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd9bd3cb640 1 -- 172.21.15.129:0/4252443951 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4032810 con 0x7fd9b80c6ec0 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/4252443951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 msgr2=0x7fd9b80c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/4252443951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 0x7fd9b80c5e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9b806b190 tx=0x7fd9b402a230 comp rx=0 tx=0).stop 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/4252443951 shutdown_connections 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/4252443951 >> v2:172.21.15.129:3300/0 conn(0x7fd9b806e4b0 0x7fd9b8163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/4252443951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 0x7fd9b80c5e60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/4252443951 >> 172.21.15.129:0/4252443951 conn(0x7fd9b80c4330 msgr2=0x7fd9b80c4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f1bcd59a640 1 -- 172.21.15.129:0/2938356752 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bb80039d0 2024-04-25T01:01:57.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fa8037fe640 1 --2- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc03f160 0x7fa7fc047cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40097a0 tx=0x7fa7f402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b471486620cde5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f616ce00640 1 -- 172.21.15.129:0/2429968926 shutdown_connections 2024-04-25T01:01:57.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f616ce00640 1 --2- 172.21.15.129:0/2429968926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600e2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f616ce00640 1 --2- 172.21.15.129:0/2429968926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61600071a0 0x7f6160007580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.916+0000 7fd3760cd640 1 -- 172.21.15.129:0/691101888 shutdown_connections 2024-04-25T01:01:57.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f616ce00640 1 -- 172.21.15.129:0/2429968926 >> 172.21.15.129:0/2429968926 conn(0x7f616004c650 msgr2=0x7f616004ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd3760cd640 1 -- 172.21.15.129:0/691101888 wait complete. 2024-04-25T01:01:57.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/1411580419 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 msgr2=0x7fbaa80ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fbaaf9e9640 1 --2- 172.21.15.129:0/1411580419 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80ab870 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40097a0 tx=0x7fbaa402a060 comp rx=0 tx=0).stop 2024-04-25T01:01:57.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/2938356752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb80039d0 msgr2=0x7f1bb8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f1bd4f1d640 1 --2- 172.21.15.129:0/2938356752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb80039d0 0x7f1bb8003e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80044d0 tx=0x7f1bb8005830 comp rx=0 tx=0).stop 2024-04-25T01:01:57.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f36777fe640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3668004c80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f366c009a90 tx=0x7f3668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b967ef77c7bc52e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fa801ffb640 1 -- 172.21.15.129:0/2618330871 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4037070 con 0x7fa7fc03f160 2024-04-25T01:01:57.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fa801ffb640 1 -- 172.21.15.129:0/2618330871 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4032440 con 0x7fa7fc03f160 2024-04-25T01:01:57.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fa801ffb640 1 -- 172.21.15.129:0/2618330871 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4032810 con 0x7fa7fc03f160 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/1411580419 shutdown_connections 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fbaaf9e9640 1 --2- 172.21.15.129:0/1411580419 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80ab870 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fbaaf9e9640 1 --2- 172.21.15.129:0/1411580419 >> v2:172.21.15.129:3300/0 conn(0x7fbaa8156cb0 0x7fbaa80c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/1411580419 >> 172.21.15.129:0/1411580419 conn(0x7fbaa80a9880 msgr2=0x7fbaa80a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 -- 172.21.15.129:0/151033856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041800a8b0 msgr2=0x7f041800ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 --2- 172.21.15.129:0/151033856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041800a8b0 0x7f041800ad00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04200502b0 tx=0x7f04200a5a10 comp rx=0 tx=0).stop 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 -- 172.21.15.129:0/151033856 shutdown_connections 2024-04-25T01:01:57.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 --2- 172.21.15.129:0/151033856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f041800a8b0 0x7f041800ad00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 -- 172.21.15.129:0/151033856 >> 172.21.15.129:0/151033856 conn(0x7f0418048250 msgr2=0x7f0418048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c036070 con 0x7f3668004800 2024-04-25T01:01:57.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366c031400 con 0x7f3668004800 2024-04-25T01:01:57.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f3676ffd640 1 -- 172.21.15.129:0/1426524795 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c009690 con 0x7f3668004800 2024-04-25T01:01:57.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/4252443951 shutdown_connections 2024-04-25T01:01:57.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/4252443951 wait complete. 2024-04-25T01:01:57.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 -- 172.21.15.129:0/151033856 shutdown_connections 2024-04-25T01:01:57.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7f1bced9d640 1 -- 172.21.15.129:0/2938356752 reap_dead start 2024-04-25T01:01:57.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/2938356752 shutdown_connections 2024-04-25T01:01:57.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/2938356752 >> 172.21.15.129:0/2938356752 conn(0x7f1bd00c4410 msgr2=0x7f1bd00c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/2938356752 shutdown_connections 2024-04-25T01:01:57.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/2938356752 wait complete. 2024-04-25T01:01:57.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f616ce00640 1 -- 172.21.15.129:0/2429968926 shutdown_connections 2024-04-25T01:01:57.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f616ce00640 1 -- 172.21.15.129:0/2429968926 wait complete. 2024-04-25T01:01:57.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/1411580419 shutdown_connections 2024-04-25T01:01:57.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.917+0000 7f041ffff640 1 -- 172.21.15.129:0/151033856 wait complete. 2024-04-25T01:01:57.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7fd9bf3cf640 1 Processor -- start 2024-04-25T01:01:57.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7fd3760cd640 1 Processor -- start 2024-04-25T01:01:57.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7fd9bf3cf640 1 -- start start 2024-04-25T01:01:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.918+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/1411580419 wait complete. 2024-04-25T01:01:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 -- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc03f160 msgr2=0x7fa7fc047cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc03f160 0x7fa7fc047cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40097a0 tx=0x7fa7f402a180 comp rx=0 tx=0).stop 2024-04-25T01:01:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f616ce00640 1 Processor -- start 2024-04-25T01:01:57.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f616ce00640 1 -- start start 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 -- 172.21.15.129:0/1426524795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 msgr2=0x7f3668004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3668004c80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f366c009a90 tx=0x7f3668005830 comp rx=0 tx=0).stop 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 -- 172.21.15.129:0/1426524795 shutdown_connections 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 --2- 172.21.15.129:0/1426524795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3668004c80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 -- 172.21.15.129:0/1426524795 >> 172.21.15.129:0/1426524795 conn(0x7f36780c6740 msgr2=0x7f36780c6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f041ffff640 1 Processor -- start 2024-04-25T01:01:57.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 -- 172.21.15.129:0/1426524795 shutdown_connections 2024-04-25T01:01:57.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7f367e929640 1 -- 172.21.15.129:0/1426524795 wait complete. 2024-04-25T01:01:57.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 -- 172.21.15.129:0/2618330871 shutdown_connections 2024-04-25T01:01:57.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc03f160 0x7fa7fc047cb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2618330871 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc0dc020 0x7fa7fc03ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 -- 172.21.15.129:0/2618330871 >> 172.21.15.129:0/2618330871 conn(0x7fa7fc006310 msgr2=0x7fa7fc006720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:57.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fa8095ab640 1 -- 172.21.15.129:0/2618330871 shutdown_connections 2024-04-25T01:01:57.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f1bd4f1d640 1 Processor -- start 2024-04-25T01:01:57.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fbaaf9e9640 1 Processor -- start 2024-04-25T01:01:57.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f1bd4f1d640 1 -- start start 2024-04-25T01:01:57.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fbaaf9e9640 1 -- start start 2024-04-25T01:01:57.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.919+0000 7fd3760cd640 1 -- start start 2024-04-25T01:01:57.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd3760cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3680073d0 0x7fd368030b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd3760cd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680dbcd0 0x7fd3680dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd3760cd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3680df170 0x7fd3681757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fa8095ab640 1 -- 172.21.15.129:0/2618330871 wait complete. 2024-04-25T01:01:57.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd3760cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3680dc0b0 con 0x7fd3680073d0 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61600071a0 0x7f6160043950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616000e9b0 0x7f61600da830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616ce00640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600dad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616ce00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f616004adb0 con 0x7f616000e9b0 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616ce00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f616004aad0 con 0x7f61600071a0 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616ce00640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f616004af30 con 0x7f616004db40 2024-04-25T01:01:57.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61600071a0 0x7f6160043950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61600071a0 0x7f6160043950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54472/0 (socket says 172.21.15.129:54472) 2024-04-25T01:01:57.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f616659c640 1 -- 172.21.15.129:0/1547298315 learned_addr learned my addr 172.21.15.129:0/1547298315 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd36f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3680073d0 0x7fd368030b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f6165d9b640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616000e9b0 0x7f61600da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd3760cd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd368047d70 con 0x7fd3680dbcd0 2024-04-25T01:01:57.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd3760cd640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd3680df5e0 con 0x7fd3680df170 2024-04-25T01:01:57.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7f041ffff640 1 -- start start 2024-04-25T01:01:57.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd36f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3680073d0 0x7fd368030b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38476/0 (socket says 172.21.15.129:38476) 2024-04-25T01:01:57.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd36f7fe640 1 -- 172.21.15.129:0/372290477 learned_addr learned my addr 172.21.15.129:0/372290477 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.920+0000 7fd36ffff640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3680df170 0x7fd3681757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f616659c640 1 -- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 msgr2=0x7f61600dad70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f616659c640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f616004db40 0x7f61600dad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f616659c640 1 -- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616000e9b0 msgr2=0x7f61600da830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f616659c640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f616000e9b0 0x7f61600da830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f616659c640 1 -- 172.21.15.129:0/1547298315 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61600d1f10 con 0x7f61600071a0 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f616659c640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61600071a0 0x7f6160043950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f615c02a770 tx=0x7f615c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f041800a8b0 0x7f0418044190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04180474a0 0x7f041804a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04180dbce0 0x7f04180431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041e7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04180474a0 0x7f041804a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041effd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f041800a8b0 0x7f0418044190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041e7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04180474a0 0x7f041804a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54490/0 (socket says 172.21.15.129:54490) 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041effd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f041800a8b0 0x7f0418044190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47616/0 (socket says 172.21.15.129:47616) 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041ffff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f041800b240 con 0x7f04180dbce0 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041ffff640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0418047d80 con 0x7f04180474a0 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041ffff640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f04180dc0c0 con 0x7f041800a8b0 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041e7fc640 1 -- 172.21.15.129:0/3643903362 learned_addr learned my addr 172.21.15.129:0/3643903362 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fa8095ab640 1 Processor -- start 2024-04-25T01:01:57.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaaf9e9640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaaf9e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa8156cb0 0x7fbaa814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaaf9e9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa8159790 0x7fbaa814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaaf9e9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80aaf80 con 0x7fbaa8156cb0 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaaf9e9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80aaca0 con 0x7fbaa8159790 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaaf9e9640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa806dc60 con 0x7fbaa80ab410 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaad785640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaad785640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47630/0 (socket says 172.21.15.129:47630) 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fbaad785640 1 -- 172.21.15.129:0/3702735316 learned_addr learned my addr 172.21.15.129:0/3702735316 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36effd640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680dbcd0 0x7fd3680dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36ffff640 1 -- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680dbcd0 msgr2=0x7fd3680dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36ffff640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3680dbcd0 0x7fd3680dd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36ffff640 1 -- 172.21.15.129:0/372290477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3680073d0 msgr2=0x7fd368030b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36ffff640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3680073d0 0x7fd368030b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36ffff640 1 -- 172.21.15.129:0/372290477 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36802e870 con 0x7fd3680df170 2024-04-25T01:01:57.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f041f7fe640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04180dbce0 0x7f04180431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fa8095ab640 1 -- start start 2024-04-25T01:01:57.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7fd36ffff640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3680df170 0x7fd3681757c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd370068850 tx=0x7fd370073750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaacf84640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa8156cb0 0x7fbaa814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaadf86640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa8159790 0x7fbaa814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c037050 con 0x7f61600071a0 2024-04-25T01:01:57.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaad785640 1 -- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa8159790 msgr2=0x7fbaa814ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaad785640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbaa8159790 0x7fbaa814ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaad785640 1 -- 172.21.15.129:0/3702735316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa8156cb0 msgr2=0x7fbaa814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaad785640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbaa8156cb0 0x7fbaa814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fbaad785640 1 -- 172.21.15.129:0/3702735316 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa81effc0 con 0x7fbaa80ab410 2024-04-25T01:01:57.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370072590 con 0x7fd3680df170 2024-04-25T01:01:57.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37008f030 con 0x7fd3680df170 2024-04-25T01:01:57.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bd4f1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bd00ad070 0x7f1bd014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61600d9d50 con 0x7f61600071a0 2024-04-25T01:01:57.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61600da310 con 0x7f61600071a0 2024-04-25T01:01:57.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c043070 con 0x7f61600071a0 2024-04-25T01:01:57.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c030bd0 con 0x7f61600071a0 2024-04-25T01:01:57.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bd4f1d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bd01573c0 0x7f1bd00aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bd4f1d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd00ab490 0x7f1bd01ea060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f041e7fc640 1 -- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f041800a8b0 msgr2=0x7f0418044190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bd4f1d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd0153560 con 0x7f1bd01573c0 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bd4f1d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd0153bb0 con 0x7f1bd00ad070 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bd4f1d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd0153840 con 0x7f1bd00ab490 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bce59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bd00ad070 0x7f1bd014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bcdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bd01573c0 0x7f1bd00aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bcdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bd01573c0 0x7f1bd00aaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38488/0 (socket says 172.21.15.129:38488) 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.921+0000 7f1bcdd9b640 1 -- 172.21.15.129:0/3833858343 learned_addr learned my addr 172.21.15.129:0/3833858343 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bced9d640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd00ab490 0x7f1bd01ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bce59c640 1 -- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd00ab490 msgr2=0x7f1bd01ea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bce59c640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1bd00ab490 0x7f1bd01ea060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bce59c640 1 -- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bd01573c0 msgr2=0x7f1bd00aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fd9bf3cf640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9b806e0d0 0x7fd9b8150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f367e929640 1 Processor -- start 2024-04-25T01:01:57.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f367e929640 1 -- start start 2024-04-25T01:01:57.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fa8095ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc004b30 0x7fa7fc041510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fd9bf3cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b806e4b0 0x7fd9b814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bce59c640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bd01573c0 0x7f1bd00aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bce59c640 1 -- 172.21.15.129:0/3833858343 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd00aa790 con 0x7f1bd00ad070 2024-04-25T01:01:57.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7fd9be3cd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9b806e0d0 0x7fd9b8150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9be3cd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9b806e0d0 0x7fd9b8150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47656/0 (socket says 172.21.15.129:47656) 2024-04-25T01:01:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f041e7fc640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f041800a8b0 0x7f0418044190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f041e7fc640 1 -- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04180dbce0 msgr2=0x7f04180431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f041e7fc640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04180dbce0 0x7f04180431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f1bcdd9b640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bd01573c0 0x7f1bd00aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f041e7fc640 1 -- 172.21.15.129:0/3643903362 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418041e80 con 0x7f04180474a0 2024-04-25T01:01:57.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.922+0000 7f041f7fe640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04180dbce0 0x7f04180431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa802ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc004b30 0x7fa7fc041510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa802ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc004b30 0x7fa7fc041510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38520/0 (socket says 172.21.15.129:38520) 2024-04-25T01:01:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9be3cd640 1 -- 172.21.15.129:0/2958363291 learned_addr learned my addr 172.21.15.129:0/2958363291 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa8095ab640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc03f160 0x7fa7fc1750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 0x7fd9b81f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f041e7fc640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04180474a0 0x7f041804a660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f041000c450 tx=0x7f041000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa802ffd640 1 -- 172.21.15.129:0/2326795178 learned_addr learned my addr 172.21.15.129:0/2326795178 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370072890 con 0x7fd3680df170 2024-04-25T01:01:57.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8159ab0 con 0x7fd9b806e4b0 2024-04-25T01:01:57.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8157040 con 0x7fd9b80c6ec0 2024-04-25T01:01:57.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b8159d90 con 0x7fd9b806e0d0 2024-04-25T01:01:57.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd3760cd640 1 -- 172.21.15.129:0/372290477 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3681779f0 con 0x7fd3680df170 2024-04-25T01:01:57.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410017070 con 0x7f04180474a0 2024-04-25T01:01:57.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0410005ce0 con 0x7f04180474a0 2024-04-25T01:01:57.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410017070 con 0x7f04180474a0 2024-04-25T01:01:57.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd3760cd640 1 -- 172.21.15.129:0/372290477 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3680dd920 con 0x7fd3680df170 2024-04-25T01:01:57.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fbaad785640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80a54b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00e810 tx=0x7fba9c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9bebce640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 0x7fd9b81f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc0dc020 0x7fa7fc178580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc00b770 con 0x7fa7fc004b30 2024-04-25T01:01:57.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0481f0 con 0x7fa7fc0dc020 2024-04-25T01:01:57.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc00b8f0 con 0x7fa7fc03f160 2024-04-25T01:01:57.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f614c001030 con 0x7f61600071a0 2024-04-25T01:01:57.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04180429d0 con 0x7f04180474a0 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0418045810 con 0x7f04180474a0 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f367e929640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3678157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f3677fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3678157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f3677fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3678157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38544/0 (socket says 172.21.15.129:38544) 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f367e929640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36780ab230 0x7f36781f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f3677fff640 1 -- 172.21.15.129:0/2290455721 learned_addr learned my addr 172.21.15.129:0/2290455721 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f367e929640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36781f1340 0x7f36781fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36780c3b10 con 0x7f3668004800 2024-04-25T01:01:57.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36781596b0 con 0x7f36781f1340 2024-04-25T01:01:57.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f36780c3c90 con 0x7f36780ab230 2024-04-25T01:01:57.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fa8027fc640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc03f160 0x7fa7fc1750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa8037fe640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc0dc020 0x7fa7fc178580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f367cec6640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36781f1340 0x7f36781fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.923+0000 7fd9bdbcc640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b806e4b0 0x7fd9b814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fd9be3cd640 1 -- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 msgr2=0x7fd9b81f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fd9be3cd640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9b80c6ec0 0x7fd9b81f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fd9be3cd640 1 -- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b806e4b0 msgr2=0x7fd9b814f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fd9be3cd640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9b806e4b0 0x7fd9b814f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fd9be3cd640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b8151810 con 0x7fd9b806e0d0 2024-04-25T01:01:57.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f1bce59c640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bd00ad070 0x7f1bd014cf70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80057e0 tx=0x7f1bc8003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f36777fe640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36780ab230 0x7f36781f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f3677fff640 1 -- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36780ab230 msgr2=0x7f36781f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f3677fff640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f36780ab230 0x7f36781f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f3677fff640 1 -- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36781f1340 msgr2=0x7f36781fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f3677fff640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36781f1340 0x7f36781fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f3677fff640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36781f8190 con 0x7f3668004800 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f367cec6640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36781f1340 0x7f36781fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:01:57.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037050 con 0x7f1bd00ad070 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa802ffd640 1 -- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc03f160 msgr2=0x7fa7fc1750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa802ffd640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa7fc03f160 0x7fa7fc1750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa802ffd640 1 -- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc0dc020 msgr2=0x7fa7fc178580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa802ffd640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7fc0dc020 0x7fa7fc178580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa802ffd640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc041a50 con 0x7fa7fc004b30 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8043070 con 0x7f1bd00ad070 2024-04-25T01:01:57.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/3833858343 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd01e7710 con 0x7f1bd00ad070 2024-04-25T01:01:57.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7f3677fff640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3678157400 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f367000fb40 tx=0x7f3670002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c017070 con 0x7fbaa80ab410 2024-04-25T01:01:57.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fd9be3cd640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9b806e0d0 0x7fd9b8150f40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000cb40 tx=0x7fd9b000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fa802ffd640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc004b30 0x7fa7fc041510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa80406a060 tx=0x7fa804073750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7fd3760cd640 1 -- 172.21.15.129:0/372290477 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd36802eb50 con 0x7fd3680df170 2024-04-25T01:01:57.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7f1bd4f1d640 1 -- 172.21.15.129:0/3833858343 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd00bcae0 con 0x7f1bd00ad070 2024-04-25T01:01:57.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.924+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa80a8ea0 con 0x7fbaa80ab410 2024-04-25T01:01:57.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa80a59f0 con 0x7fbaa80ab410 2024-04-25T01:01:57.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f615c048460 con 0x7f61600071a0 2024-04-25T01:01:57.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c00c8d0 con 0x7fbaa80ab410 2024-04-25T01:01:57.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80040a0 con 0x7f1bd00ad070 2024-04-25T01:01:57.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbaa81efd70 con 0x7fbaa80ab410 2024-04-25T01:01:57.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd370072a30 con 0x7fd3680df170 2024-04-25T01:01:57.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f6156ffd640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f614806bf30 0x7f614806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fd35bfff640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35406bf30 0x7fd35406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f041001a070 con 0x7f04180474a0 2024-04-25T01:01:57.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0400001030 con 0x7f04180474a0 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f04077fe640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f040c06c230 0x7f040c06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd370106be0 con 0x7fd3680df170 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0017070 con 0x7fd9b806e0d0 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b000aca0 con 0x7fd9b806e0d0 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0012620 con 0x7fd9b806e0d0 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b81f0110 con 0x7fd9b806e0d0 2024-04-25T01:01:57.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b8151480 con 0x7fd9b806e0d0 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f6165d9b640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f614806bf30 0x7f614806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f615c0abef0 con 0x7f61600071a0 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bc803c8b0 con 0x7f1bd00ad070 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bd0075550 con 0x7f1bd00ad070 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0410005ae0 con 0x7f04180474a0 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f04077fe640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f040c076500 0x7f040c0789a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 --> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f040c079070 con 0x7f040c076500 2024-04-25T01:01:57.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f041effd640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f040c06c230 0x7f040c06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804069320 con 0x7fa7fc004b30 2024-04-25T01:01:57.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa804072590 con 0x7fa7fc004b30 2024-04-25T01:01:57.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804072940 con 0x7fa7fc004b30 2024-04-25T01:01:57.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fd35bfff640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7fd354075630 0x7fd354077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 --> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd37008f030 con 0x7fd354075630 2024-04-25T01:01:57.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f1bbeffd640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1bb006bf00 0x7f1bb006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc175610 con 0x7fa7fc004b30 2024-04-25T01:01:57.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c017070 con 0x7fbaa80ab410 2024-04-25T01:01:57.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc0d9920 con 0x7fa7fc004b30 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670003bb0 con 0x7f3668004800 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3670018070 con 0x7f3668004800 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.925+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367000d790 con 0x7f3668004800 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36781f73d0 con 0x7f3668004800 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.926+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36781f7990 con 0x7f3668004800 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fd36f7fe640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35406bf30 0x7fd35406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1bc80aba90 con 0x7f1bd00ad070 2024-04-25T01:01:57.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f041f7fe640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f040c076500 0x7f040c0789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0410054880 con 0x7f04180474a0 2024-04-25T01:01:57.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f041f7fe640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f040c076500 0x7f040c0789a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f6156ffd640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7f61480756a0 0x7f6148077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f367806cad0 con 0x7f3668004800 2024-04-25T01:01:57.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd3700d0e80 con 0x7fd3680df170 2024-04-25T01:01:57.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7fd36effd640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7fd354075630 0x7fd354077a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7fc005810 con 0x7fa7fc004b30 2024-04-25T01:01:57.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 --> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f615c02be50 con 0x7f61480756a0 2024-04-25T01:01:57.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.927+0000 7f1bcdd9b640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1bb006bf00 0x7f1bb006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f1bbeffd640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f1bb00755f0 0x7f1bb0077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 --> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bc802ad80 con 0x7f1bb00755f0 2024-04-25T01:01:57.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f1bcdd9b640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1bb006bf00 0x7f1bb006e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bd01577a0 tx=0x7f1bc4001ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f6165d9b640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f614806bf30 0x7f614806e3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6168050a50 tx=0x7f616806abb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bc80b2050 con 0x7f1bd00ad070 2024-04-25T01:01:57.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f615c0ae550 con 0x7f61600071a0 2024-04-25T01:01:57.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9b81576b0 con 0x7fd9b806e0d0 2024-04-25T01:01:57.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.928+0000 7f6166d9d640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7f61480756a0 0x7f6148077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f041effd640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f040c06c230 0x7f040c06e6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0414006fd0 tx=0x7f0414009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f6166d9d640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7f61480756a0 0x7f6148077ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f367000d930 con 0x7f3668004800 2024-04-25T01:01:57.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9b001a070 con 0x7fd9b806e0d0 2024-04-25T01:01:57.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f1bced9d640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f1bb00755f0 0x7f1bb0077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== osd.3 v2:172.21.15.129:6808/4291613316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f040c079070 con 0x7f040c076500 2024-04-25T01:01:57.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa804072ae0 con 0x7fa7fc004b30 2024-04-25T01:01:57.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9aeffd640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd99c06bf00 0x7fd99c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f3674ff9640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f365c06be80 0x7f365c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f367008b3e0 con 0x7f3668004800 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f36777fe640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f365c06be80 0x7f365c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f3674ff9640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f365c075600 0x7f365c077a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3670016e50 con 0x7f365c075600 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3670092050 con 0x7f3668004800 2024-04-25T01:01:57.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7fd36effd640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7fd354075630 0x7fd354077a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7fd36f7fe640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35406bf30 0x7fd35406e3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd364006fd0 tx=0x7fd364008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:57.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd9b008af40 con 0x7fd9b806e0d0 2024-04-25T01:01:57.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.929+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba9c01a070 con 0x7fbaa80ab410 2024-04-25T01:01:58.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9bdbcc640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd99c06bf00 0x7fd99c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9aeffd640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7fd99c075680 0x7fd99c077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd99c078190 con 0x7fd99c075680 2024-04-25T01:01:58.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9b008d5a0 con 0x7fd9b806e0d0 2024-04-25T01:01:58.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fa7eb7fe640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa7f006bf30 0x7fa7f006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fba99ffb640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba8c06c230 0x7fba8c06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== osd.1 v2:172.21.15.129:6816/3197437953 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f615c02be50 con 0x7f61480756a0 2024-04-25T01:01:58.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== osd.7 v2:172.21.15.171:6800/3134257063 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd37008f030 con 0x7fd354075630 2024-04-25T01:01:58.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba9c00c6d0 con 0x7fbaa80ab410 2024-04-25T01:01:58.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa804106c40 con 0x7fa7fc004b30 2024-04-25T01:01:58.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fba99ffb640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7fba8c0758a0 0x7fba8c077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fa7eb7fe640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7fa7f0075620 0x7fa7f0077a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7f0078130 con 0x7fa7f0075620 2024-04-25T01:01:58.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa80410d020 con 0x7fa7fc004b30 2024-04-25T01:01:58.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7fd9bebce640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7fd99c075680 0x7fd99c077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f1bced9d640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f1bb00755f0 0x7f1bb0077a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f367cec6640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f365c075600 0x7f365c077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.930+0000 7f36777fe640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f365c06be80 0x7f365c06e340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f366c033000 tx=0x7f366c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 --> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba8c0783b0 con 0x7fba8c0758a0 2024-04-25T01:01:58.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba9c08adb0 con 0x7fbaa80ab410 2024-04-25T01:01:58.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fbaacf84640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba8c06c230 0x7fba8c06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7f367cec6640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f365c075600 0x7f365c077a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fd9bebce640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7fd99c075680 0x7fd99c077ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fbaadf86640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7fba8c0758a0 0x7fba8c077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fbaadf86640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7fba8c0758a0 0x7fba8c077ce0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fa8037fe640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7fa7f0075620 0x7fa7f0077a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fa8027fc640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa7f006bf30 0x7fa7f006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.931+0000 7fd9bdbcc640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd99c06bf00 0x7fd99c06e3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4034000 tx=0x7fd9b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7fa8037fe640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7fa7f0075620 0x7fa7f0077a60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7fa8027fc640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa7f006bf30 0x7fa7f006e3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0d5ad0 tx=0x7fa7f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== osd.2 v2:172.21.15.129:6824/1981636327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bc802ad80 con 0x7f1bb00755f0 2024-04-25T01:01:58.006 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== osd.4 v2:172.21.15.171:6808/4119213215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3670016e50 con 0x7f365c075600 2024-04-25T01:01:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7fbaacf84640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba8c06c230 0x7fba8c06e6f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4034000 tx=0x7fbaa402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== osd.0 v2:172.21.15.129:6800/1195619493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd99c078190 con 0x7fd99c075680 2024-04-25T01:01:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== osd.6 v2:172.21.15.171:6824/2873795804 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba8c0783b0 con 0x7fba8c0758a0 2024-04-25T01:01:58.007 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.932+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== osd.5 v2:172.21.15.171:6809/486235959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7f0078130 con 0x7fa7f0075620 2024-04-25T01:01:58.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.962+0000 7fd3760cd640 1 -- 172.21.15.129:0/372290477 --> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd368005380 con 0x7fd354075630 2024-04-25T01:01:58.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.963+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 --> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd9b815c940 con 0x7fd99c075680 2024-04-25T01:01:58.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.963+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 --> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0400002dd0 con 0x7f040c076500 2024-04-25T01:01:58.008 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.963+0000 7fd35bfff640 1 -- 172.21.15.129:0/372290477 <== osd.7 v2:172.21.15.171:6800/3134257063 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd368005380 con 0x7fd354075630 2024-04-25T01:01:58.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd9aeffd640 1 -- 172.21.15.129:0/2958363291 <== osd.0 v2:172.21.15.129:6800/1195619493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd9b815c940 con 0x7fd99c075680 2024-04-25T01:01:58.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7f04077fe640 1 -- 172.21.15.129:0/3643903362 <== osd.3 v2:172.21.15.129:6808/4291613316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0400002dd0 con 0x7f040c076500 2024-04-25T01:01:58.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 --> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f614c002dd0 con 0x7f61480756a0 2024-04-25T01:01:58.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7fd354075630 msgr2=0x7fd354077a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd359ffb640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7fd354075630 0x7fd354077a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7fd99c075680 msgr2=0x7fd99c077ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7fd99c075680 0x7fd99c077ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 --> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbaa81f02a0 con 0x7fba8c0758a0 2024-04-25T01:01:58.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 --> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1bd0150bb0 con 0x7f1bb00755f0 2024-04-25T01:01:58.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f040c076500 msgr2=0x7f040c0789a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd99c06bf00 msgr2=0x7fd99c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7f041ffff640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f040c076500 0x7f040c0789a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.964+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd99c06bf00 0x7fd99c06e3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4034000 tx=0x7fd9b402e000 comp rx=0 tx=0).stop 2024-04-25T01:01:58.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9b806e0d0 msgr2=0x7fd9b8150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bf3cf640 1 --2- 172.21.15.129:0/2958363291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9b806e0d0 0x7fd9b8150f40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000cb40 tx=0x7fd9b000a060 comp rx=0 tx=0).stop 2024-04-25T01:01:58.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bebce640 1 -- 172.21.15.129:0/2958363291 reap_dead start 2024-04-25T01:01:58.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f6156ffd640 1 -- 172.21.15.129:0/1547298315 <== osd.1 v2:172.21.15.129:6816/3197437953 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f614c002dd0 con 0x7f61480756a0 2024-04-25T01:01:58.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f040c06c230 msgr2=0x7f040c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35406bf30 msgr2=0x7fd35406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 shutdown_connections 2024-04-25T01:01:58.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f040c06c230 0x7f040c06e6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0414006fd0 tx=0x7f0414009060 comp rx=0 tx=0).stop 2024-04-25T01:01:58.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35406bf30 0x7fd35406e3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd364006fd0 tx=0x7fd364008040 comp rx=0 tx=0).stop 2024-04-25T01:01:58.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 >> 172.21.15.129:0/2958363291 conn(0x7fd9b80c4330 msgr2=0x7fd9b80c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3680df170 msgr2=0x7fd3681757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04180474a0 msgr2=0x7f041804a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 --2- 172.21.15.129:0/372290477 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3680df170 0x7fd3681757c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd370068850 tx=0x7fd370073750 comp rx=0 tx=0).stop 2024-04-25T01:01:58.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 --2- 172.21.15.129:0/3643903362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04180474a0 0x7f041804a660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f041000c450 tx=0x7f041000c910 comp rx=0 tx=0).stop 2024-04-25T01:01:58.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041f7fe640 1 -- 172.21.15.129:0/3643903362 reap_dead start 2024-04-25T01:01:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd36ffff640 1 -- 172.21.15.129:0/372290477 reap_dead start 2024-04-25T01:01:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f1bbeffd640 1 -- 172.21.15.129:0/3833858343 <== osd.2 v2:172.21.15.129:6824/1981636327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1bd0150bb0 con 0x7f1bb00755f0 2024-04-25T01:01:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 shutdown_connections 2024-04-25T01:01:58.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 >> 172.21.15.129:0/3643903362 conn(0x7f0418048250 msgr2=0x7f04180dc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 shutdown_connections 2024-04-25T01:01:58.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 >> 172.21.15.129:0/372290477 conn(0x7fd368048240 msgr2=0x7fd3680dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fba99ffb640 1 -- 172.21.15.129:0/3702735316 <== osd.6 v2:172.21.15.171:6824/2873795804 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbaa81f02a0 con 0x7fba8c0758a0 2024-04-25T01:01:58.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 shutdown_connections 2024-04-25T01:01:58.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 shutdown_connections 2024-04-25T01:01:58.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd9bf3cf640 1 -- 172.21.15.129:0/2958363291 wait complete. 2024-04-25T01:01:58.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 --> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36780753d0 con 0x7f365c075600 2024-04-25T01:01:58.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f041ffff640 1 -- 172.21.15.129:0/3643903362 wait complete. 2024-04-25T01:01:58.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 shutdown_connections 2024-04-25T01:01:58.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7f61480756a0 msgr2=0x7f6148077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7f616ce00640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7f61480756a0 0x7f6148077ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.965+0000 7fd359ffb640 1 -- 172.21.15.129:0/372290477 wait complete. 2024-04-25T01:01:58.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f1bb00755f0 msgr2=0x7f1bb0077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f1bb00755f0 0x7f1bb0077a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f3674ff9640 1 -- 172.21.15.129:0/2290455721 <== osd.4 v2:172.21.15.171:6808/4119213215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f36780753d0 con 0x7f365c075600 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 --> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7fc1780b0 con 0x7fa7f0075620 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1bb006bf00 msgr2=0x7f1bb006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1bb006bf00 0x7f1bb006e3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bd01577a0 tx=0x7f1bc4001ac0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bd00ad070 msgr2=0x7f1bd014cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 --2- 172.21.15.129:0/3833858343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1bd00ad070 0x7f1bd014cf70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80057e0 tx=0x7f1bc8003a70 comp rx=0 tx=0).stop 2024-04-25T01:01:58.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bced9d640 1 -- 172.21.15.129:0/3833858343 reap_dead start 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f614806bf30 msgr2=0x7f614806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f614806bf30 0x7f614806e3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6168050a50 tx=0x7f616806abb0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61600071a0 msgr2=0x7f6160043950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 --2- 172.21.15.129:0/1547298315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61600071a0 0x7f6160043950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f615c02a770 tx=0x7f615c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f6166d9d640 1 -- 172.21.15.129:0/1547298315 reap_dead start 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 shutdown_connections 2024-04-25T01:01:58.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 >> 172.21.15.129:0/1547298315 conn(0x7f616004c650 msgr2=0x7f61600dfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7fba8c0758a0 msgr2=0x7fba8c077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 shutdown_connections 2024-04-25T01:01:58.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaaf9e9640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7fba8c0758a0 0x7fba8c077ce0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 shutdown_connections 2024-04-25T01:01:58.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 >> 172.21.15.129:0/3833858343 conn(0x7f1bd00c4410 msgr2=0x7f1bd0151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f616ce00640 1 -- 172.21.15.129:0/1547298315 wait complete. 2024-04-25T01:01:58.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f365c075600 msgr2=0x7f365c077a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f365c075600 0x7f365c077a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.025 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fa7eb7fe640 1 -- 172.21.15.129:0/2326795178 <== osd.5 v2:172.21.15.171:6809/486235959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa7fc1780b0 con 0x7fa7f0075620 2024-04-25T01:01:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 shutdown_connections 2024-04-25T01:01:58.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f1bbcff9640 1 -- 172.21.15.129:0/3833858343 wait complete. 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f365c06be80 msgr2=0x7f365c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f365c06be80 0x7f365c06e340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f366c033000 tx=0x7f366c008030 comp rx=0 tx=0).stop 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 msgr2=0x7f3678157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 --2- 172.21.15.129:0/2290455721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668004800 0x7f3678157400 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f367000fb40 tx=0x7f3670002150 comp rx=0 tx=0).stop 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367cec6640 1 -- 172.21.15.129:0/2290455721 reap_dead start 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 shutdown_connections 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 >> 172.21.15.129:0/2290455721 conn(0x7f36780c6740 msgr2=0x7f36780c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba8c06c230 msgr2=0x7fba8c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaaf9e9640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba8c06c230 0x7fba8c06e6f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4034000 tx=0x7fbaa402e000 comp rx=0 tx=0).stop 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 msgr2=0x7fbaa80a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaaf9e9640 1 --2- 172.21.15.129:0/3702735316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbaa80ab410 0x7fbaa80a54b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00e810 tx=0x7fba9c00ecd0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.966+0000 7fbaadf86640 1 -- 172.21.15.129:0/3702735316 reap_dead start 2024-04-25T01:01:58.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 shutdown_connections 2024-04-25T01:01:58.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7fa7f0075620 msgr2=0x7fa7f0077a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 shutdown_connections 2024-04-25T01:01:58.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7fa7f0075620 0x7fa7f0077a60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 >> 172.21.15.129:0/3702735316 conn(0x7fbaa80a9880 msgr2=0x7fbaa8153650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7f367e929640 1 -- 172.21.15.129:0/2290455721 wait complete. 2024-04-25T01:01:58.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa7f006bf30 msgr2=0x7fa7f006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa7f006bf30 0x7fa7f006e3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0d5ad0 tx=0x7fa7f8009040 comp rx=0 tx=0).stop 2024-04-25T01:01:58.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc004b30 msgr2=0x7fa7fc041510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 --2- 172.21.15.129:0/2326795178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc004b30 0x7fa7fc041510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa80406a060 tx=0x7fa804073750 comp rx=0 tx=0).stop 2024-04-25T01:01:58.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8037fe640 1 -- 172.21.15.129:0/2326795178 reap_dead start 2024-04-25T01:01:58.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 shutdown_connections 2024-04-25T01:01:58.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.967+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 >> 172.21.15.129:0/2326795178 conn(0x7fa7fc006310 msgr2=0x7fa7fc0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.968+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 shutdown_connections 2024-04-25T01:01:58.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.968+0000 7fa8095ab640 1 -- 172.21.15.129:0/2326795178 wait complete. 2024-04-25T01:01:58.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.968+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 shutdown_connections 2024-04-25T01:01:58.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:57.968+0000 7fbaaf9e9640 1 -- 172.21.15.129:0/3702735316 wait complete. 2024-04-25T01:01:58.031 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.032 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.033 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T01:01:58.033 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T01:01:58.034 INFO:teuthology.orchestra.run.smithi129.stdout:55834574859 2024-04-25T01:01:58.034 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T01:01:58.035 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.035 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T01:01:58.036 INFO:teuthology.orchestra.run.smithi129.stdout:55834574859 2024-04-25T01:01:58.036 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T01:01:58.037 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.037 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T01:01:58.038 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.038 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T01:01:58.039 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:01:58.039 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T01:01:58.060 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:01:58.083 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:01:58.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.243+0000 7ffa0b509640 1 Processor -- start 2024-04-25T01:01:58.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.243+0000 7ffa0b509640 1 -- start start 2024-04-25T01:01:58.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.244+0000 7f13d359e640 1 Processor -- start 2024-04-25T01:01:58.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.244+0000 7f13d359e640 1 -- start start 2024-04-25T01:01:58.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f4eab59e640 1 Processor -- start 2024-04-25T01:01:58.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f4eab59e640 1 -- start start 2024-04-25T01:01:58.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.244+0000 7ffa0b509640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff9fc0d3be0 0x7ff9fc0d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.244+0000 7ffa0b509640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff9fc0d6fd0 0x7ff9fc0dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7ffa0b509640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9fc046900 con 0x7ff9fc00b100 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7ffa0b509640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7ff9fc047690 con 0x7ff9fc0d3be0 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7ffa0b509640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7ff9fc0d7420 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7ffa08aa4640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff9fc0d3be0 0x7ff9fc0d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7ffa08aa4640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7ff9fc0d3be0 0x7ff9fc0d6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38552/0 (socket says 172.21.15.129:38552) 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7ffa08aa4640 1 -- 172.21.15.129:0/1663646672 learned_addr learned my addr 172.21.15.129:0/1663646672 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f13d359e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f13cc0e4520 0x7f13cc049f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f13d359e640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f13cc006250 0x7f13cc0066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f13d359e640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13cc0e7000 con 0x7f13cc050720 2024-04-25T01:01:58.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f13d359e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc0e72a0 con 0x7f13cc0e4520 2024-04-25T01:01:58.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.245+0000 7f13d359e640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f13cc0e7580 con 0x7f13cc006250 2024-04-25T01:01:58.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f13cc006250 0x7f13cc0066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7ff84e422640 1 Processor -- start 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d259c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f13cc0e4520 0x7f13cc049f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f13cc006250 0x7f13cc0066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54566/0 (socket says 172.21.15.129:54566) 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 -- 172.21.15.129:0/4274504511 learned_addr learned my addr 172.21.15.129:0/4274504511 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d1d9b640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f13cc050720 0x7f13cc00edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54044/0 (socket says 172.21.15.129:54044) 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f4eab59e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f4eac156cd0 0x7f4eac0ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f4eab59e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac0aa720 0x7f4eac0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f4eab59e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eac06dc40 con 0x7f4eac0c6be0 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f4eab59e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f4eac0abc10 con 0x7f4eac156cd0 2024-04-25T01:01:58.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f4eab59e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac06df20 con 0x7f4eac0aa720 2024-04-25T01:01:58.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7ff84e422640 1 -- start start 2024-04-25T01:01:58.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7ffa08aa4640 1 -- 172.21.15.129:0/1663646672 >> v2:172.21.15.171:3300/0 conn(0x7ff9fc0d6fd0 msgr2=0x7ff9fc0dacd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:01:58.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7ffa08aa4640 1 --2- 172.21.15.129:0/1663646672 >> v2:172.21.15.171:3300/0 conn(0x7ff9fc0d6fd0 0x7ff9fc0dacd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7ffa08aa4640 1 -- 172.21.15.129:0/1663646672 >> v1:172.21.15.129:6789/0 conn(0x7ff9fc00b100 legacy=0x7ff9fc04aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 -- 172.21.15.129:0/4274504511 >> v2:172.21.15.129:3300/0 conn(0x7f13cc0e4520 msgr2=0x7f13cc049f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 --2- 172.21.15.129:0/4274504511 >> v2:172.21.15.129:3300/0 conn(0x7f13cc0e4520 0x7f13cc049f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 -- 172.21.15.129:0/4274504511 >> v1:172.21.15.171:6789/0 conn(0x7f13cc050720 legacy=0x7f13cc00edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7f13d2d9d640 1 -- 172.21.15.129:0/4274504511 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc0e7860 con 0x7f13cc006250 2024-04-25T01:01:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.246+0000 7ffa08aa4640 1 -- 172.21.15.129:0/1663646672 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc046be0 con 0x7ff9fc0d3be0 2024-04-25T01:01:58.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f1fcdfe4640 1 Processor -- start 2024-04-25T01:01:58.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4ea9d9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac0aa720 0x7f4eac0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4ea9d9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac0aa720 0x7f4eac0c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47670/0 (socket says 172.21.15.129:47670) 2024-04-25T01:01:58.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4ea9d9b640 1 -- 172.21.15.129:0/2215174596 learned_addr learned my addr 172.21.15.129:0/2215174596 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4eaa59c640 1 --2- 172.21.15.129:0/2215174596 >> v2:172.21.15.129:3300/0 conn(0x7f4eac156cd0 0x7f4eac0ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff84e422640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400474d0 0x7ff840006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff84e422640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff840007410 0x7ff84000a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff84e422640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff84000bb80 con 0x7ff8400dbd10 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff84e422640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7ff84000a450 con 0x7ff840007410 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff84e422640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff840048090 con 0x7ff8400474d0 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff847fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff8400dbd10 0x7ff8400df1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44728/0 (socket says 172.21.15.129:44728) 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff847fff640 1 -- 172.21.15.129:0/3267766416 learned_addr learned my addr 172.21.15.129:0/3267766416 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff8477fe640 1 --2- 172.21.15.129:0/3267766416 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400474d0 0x7ff840006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ffa08aa4640 1 --2- 172.21.15.129:0/1663646672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0d6a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa0407af10 tx=0x7ffa040ae5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af70bc3be6ebde7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7ff84c9bf640 1 --2- 172.21.15.129:0/3267766416 >> v2:172.21.15.171:3300/0 conn(0x7ff840007410 0x7ff84000a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff8477fe640 1 -- 172.21.15.129:0/3267766416 >> v2:172.21.15.171:3300/0 conn(0x7ff840007410 msgr2=0x7ff84000a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff9fbfff640 1 -- 172.21.15.129:0/1663646672 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040b5600 con 0x7ff9fc0d3be0 2024-04-25T01:01:58.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff9fbfff640 1 -- 172.21.15.129:0/1663646672 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa040b57a0 con 0x7ff9fc0d3be0 2024-04-25T01:01:58.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff9fbfff640 1 -- 172.21.15.129:0/1663646672 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040b5c10 con 0x7ff9fc0d3be0 2024-04-25T01:01:58.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4eaa59c640 1 -- 172.21.15.129:0/2215174596 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac0aa720 msgr2=0x7f4eac0c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4eaa59c640 1 --2- 172.21.15.129:0/2215174596 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac0aa720 0x7f4eac0c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4eaa59c640 1 -- 172.21.15.129:0/2215174596 >> v1:172.21.15.129:6789/0 conn(0x7f4eac0c6be0 legacy=0x7f4eac1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7fc4ad32e640 1 Processor -- start 2024-04-25T01:01:58.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7fc4ad32e640 1 -- start start 2024-04-25T01:01:58.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f4eaa59c640 1 -- 172.21.15.129:0/2215174596 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac0aae30 con 0x7f4eac156cd0 2024-04-25T01:01:58.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.247+0000 7f1fcdfe4640 1 -- start start 2024-04-25T01:01:58.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f4eaa59c640 1 --2- 172.21.15.129:0/2215174596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac156cd0 0x7f4eac0ab6d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097a0 tx=0x7f4e9c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5041616a27c0bf8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f13d2d9d640 1 --2- 172.21.15.129:0/4274504511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc006250 0x7f13cc0066c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13d407af10 tx=0x7f13d40a5bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf80032ddcd8974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fcdfe4640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 0x7f1fc000d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fcdfe4640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1fc00d8240 0x7f1fc004ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fcdfe4640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc000c890 con 0x7f1fc004bf40 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fcdfe4640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc000cb70 con 0x7f1fc00d8240 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fcdfe4640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc000ce50 con 0x7f1fc00d5970 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fccfe2640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 0x7f1fc000d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fccfe2640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 0x7f1fc000d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47678/0 (socket says 172.21.15.129:47678) 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff8477fe640 1 --2- 172.21.15.129:0/3267766416 >> v2:172.21.15.171:3300/0 conn(0x7ff840007410 0x7ff84000a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff8477fe640 1 -- 172.21.15.129:0/3267766416 >> v1:172.21.15.129:6789/0 conn(0x7ff8400dbd10 legacy=0x7ff8400df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7ff8477fe640 1 -- 172.21.15.129:0/3267766416 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84000a120 con 0x7ff8400474d0 2024-04-25T01:01:58.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fccfe2640 1 -- 172.21.15.129:0/115811633 learned_addr learned my addr 172.21.15.129:0/115811633 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.248+0000 7f1fc7fff640 1 --2- 172.21.15.129:0/115811633 >> v2:172.21.15.171:3300/0 conn(0x7f1fc00d8240 0x7f1fc004ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f4ea959a640 1 -- 172.21.15.129:0/2215174596 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c037070 con 0x7f4eac156cd0 2024-04-25T01:01:58.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7ff8477fe640 1 --2- 172.21.15.129:0/3267766416 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400474d0 0x7ff840006ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0097a0 tx=0x7ff83c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93caaefb70556a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f1fccfe2640 1 -- 172.21.15.129:0/115811633 >> v2:172.21.15.171:3300/0 conn(0x7f1fc00d8240 msgr2=0x7f1fc004ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4ad32e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 0x7fc4a00d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7ff846ffd640 1 -- 172.21.15.129:0/3267766416 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c037070 con 0x7ff8400474d0 2024-04-25T01:01:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7ff846ffd640 1 -- 172.21.15.129:0/3267766416 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c02ae50 con 0x7ff8400474d0 2024-04-25T01:01:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7ff846ffd640 1 -- 172.21.15.129:0/3267766416 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c032660 con 0x7ff8400474d0 2024-04-25T01:01:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4ad32e640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc4a004b110 0x7fc4a0046b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4ad32e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a0047050 con 0x7fc4a00d6710 2024-04-25T01:01:58.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4ad32e640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a0007730 con 0x7fc4a004b110 2024-04-25T01:01:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4ad32e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a00d4080 con 0x7fc4a000bd50 2024-04-25T01:01:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ffa0b509640 1 -- 172.21.15.129:0/1663646672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 msgr2=0x7ff9fc0d6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ffa0b509640 1 --2- 172.21.15.129:0/1663646672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0d6a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa0407af10 tx=0x7ffa040ae5a0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f4ea959a640 1 -- 172.21.15.129:0/2215174596 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9c032440 con 0x7f4eac156cd0 2024-04-25T01:01:58.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f4ea959a640 1 -- 172.21.15.129:0/2215174596 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c02ae60 con 0x7f4eac156cd0 2024-04-25T01:01:58.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f1fccfe2640 1 --2- 172.21.15.129:0/115811633 >> v2:172.21.15.171:3300/0 conn(0x7f1fc00d8240 0x7f1fc004ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f1fccfe2640 1 -- 172.21.15.129:0/115811633 >> v1:172.21.15.171:6789/0 conn(0x7f1fc004bf40 legacy=0x7f1fc0048360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f1fccfe2640 1 -- 172.21.15.129:0/115811633 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc000d130 con 0x7f1fc00d5970 2024-04-25T01:01:58.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 -- 172.21.15.129:0/2215174596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac156cd0 msgr2=0x7f4eac0ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 --2- 172.21.15.129:0/2215174596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac156cd0 0x7f4eac0ab6d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0097a0 tx=0x7f4e9c02a150 comp rx=0 tx=0).stop 2024-04-25T01:01:58.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f13d159a640 1 -- 172.21.15.129:0/4274504511 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40b3070 con 0x7f13cc006250 2024-04-25T01:01:58.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f13d159a640 1 -- 172.21.15.129:0/4274504511 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d40a6a40 con 0x7f13cc006250 2024-04-25T01:01:58.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7f13d159a640 1 -- 172.21.15.129:0/4274504511 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40ae440 con 0x7f13cc006250 2024-04-25T01:01:58.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ffa0b509640 1 -- 172.21.15.129:0/1663646672 shutdown_connections 2024-04-25T01:01:58.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ffa0b509640 1 --2- 172.21.15.129:0/1663646672 >> v2:172.21.15.171:3300/0 conn(0x7ff9fc0d6fd0 0x7ff9fc0dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f13d359e640 1 -- 172.21.15.129:0/4274504511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc006250 msgr2=0x7f13cc0066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ffa0b509640 1 --2- 172.21.15.129:0/1663646672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0d6a90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ffa0b509640 1 -- 172.21.15.129:0/1663646672 >> 172.21.15.129:0/1663646672 conn(0x7ff9fc005ec0 msgr2=0x7ff9fc0062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 -- 172.21.15.129:0/2215174596 shutdown_connections 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 --2- 172.21.15.129:0/2215174596 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac0aa720 0x7f4eac0c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 --2- 172.21.15.129:0/2215174596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac156cd0 0x7f4eac0ab6d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 -- 172.21.15.129:0/2215174596 >> 172.21.15.129:0/2215174596 conn(0x7f4eac06cc80 msgr2=0x7f4eac0a9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7fba6f59e640 1 Processor -- start 2024-04-25T01:01:58.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f1fccfe2640 1 --2- 172.21.15.129:0/115811633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 0x7f1fc000d930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009870 tx=0x7f1fb402a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77096413b6eeec21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 -- 172.21.15.129:0/2215174596 shutdown_connections 2024-04-25T01:01:58.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7fba6f59e640 1 -- start start 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ff84e422640 1 -- 172.21.15.129:0/3267766416 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400474d0 msgr2=0x7ff840006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7ff84e422640 1 --2- 172.21.15.129:0/3267766416 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400474d0 0x7ff840006ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0097a0 tx=0x7ff83c02a100 comp rx=0 tx=0).stop 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f13d359e640 1 --2- 172.21.15.129:0/4274504511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc006250 0x7f13cc0066c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13d407af10 tx=0x7f13d40a5bf0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f13d359e640 1 -- 172.21.15.129:0/4274504511 shutdown_connections 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f13d359e640 1 --2- 172.21.15.129:0/4274504511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc006250 0x7f13cc0066c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f13d359e640 1 --2- 172.21.15.129:0/4274504511 >> v2:172.21.15.129:3300/0 conn(0x7f13cc0e4520 0x7f13cc049f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f13d359e640 1 -- 172.21.15.129:0/4274504511 >> 172.21.15.129:0/4274504511 conn(0x7f13cc04fc00 msgr2=0x7f13cc050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f1fc77fe640 1 -- 172.21.15.129:0/115811633 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4037070 con 0x7f1fc00d5970 2024-04-25T01:01:58.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f1fc77fe640 1 -- 172.21.15.129:0/115811633 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb402ade0 con 0x7f1fc00d5970 2024-04-25T01:01:58.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 0x7fc4a00d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 0x7fc4a00d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47680/0 (socket says 172.21.15.129:47680) 2024-04-25T01:01:58.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.249+0000 7fc4a7fff640 1 -- 172.21.15.129:0/1763226110 learned_addr learned my addr 172.21.15.129:0/1763226110 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7fc4a7fff640 1 -- 172.21.15.129:0/1763226110 >> v2:172.21.15.171:3300/0 conn(0x7fc4a004b110 msgr2=0x7fc4a0046b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:01:58.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ffa0b509640 1 -- 172.21.15.129:0/1663646672 shutdown_connections 2024-04-25T01:01:58.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7fc4a7fff640 1 --2- 172.21.15.129:0/1763226110 >> v2:172.21.15.171:3300/0 conn(0x7fc4a004b110 0x7fc4a0046b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7fc4a7fff640 1 -- 172.21.15.129:0/1763226110 >> v1:172.21.15.129:6789/0 conn(0x7fc4a00d6710 legacy=0x7fc4a00d6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7fc4a7fff640 1 -- 172.21.15.129:0/1763226110 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a00d72b0 con 0x7fc4a000bd50 2024-04-25T01:01:58.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ff84e422640 1 -- 172.21.15.129:0/3267766416 shutdown_connections 2024-04-25T01:01:58.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ff84e422640 1 --2- 172.21.15.129:0/3267766416 >> v2:172.21.15.171:3300/0 conn(0x7ff840007410 0x7ff84000a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ff84e422640 1 --2- 172.21.15.129:0/3267766416 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400474d0 0x7ff840006ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ff84e422640 1 -- 172.21.15.129:0/3267766416 >> 172.21.15.129:0/3267766416 conn(0x7ff840048280 msgr2=0x7ff840048690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7f13d359e640 1 -- 172.21.15.129:0/4274504511 shutdown_connections 2024-04-25T01:01:58.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7f13d359e640 1 -- 172.21.15.129:0/4274504511 wait complete. 2024-04-25T01:01:58.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ff84e422640 1 -- 172.21.15.129:0/3267766416 shutdown_connections 2024-04-25T01:01:58.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ff84e422640 1 -- 172.21.15.129:0/3267766416 wait complete. 2024-04-25T01:01:58.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.250+0000 7f4eab59e640 1 -- 172.21.15.129:0/2215174596 wait complete. 2024-04-25T01:01:58.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7f1fc77fe640 1 -- 172.21.15.129:0/115811633 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4032660 con 0x7f1fc00d5970 2024-04-25T01:01:58.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/115811633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 msgr2=0x7f1fc000d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7fba6f59e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba700a7030 0x7fba700c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7fba6f59e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fba700c3150 0x7fba700c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/115811633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 0x7f1fc000d930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009870 tx=0x7f1fb402a150 comp rx=0 tx=0).stop 2024-04-25T01:01:58.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/115811633 shutdown_connections 2024-04-25T01:01:58.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/115811633 >> v2:172.21.15.171:3300/0 conn(0x7f1fc00d8240 0x7f1fc004ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/115811633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc00d5970 0x7f1fc000d930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/115811633 >> 172.21.15.129:0/115811633 conn(0x7f1fc004d9e0 msgr2=0x7f1fc004ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7fba6f59e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba70153270 con 0x7fba701506f0 2024-04-25T01:01:58.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7ffa0b509640 1 -- 172.21.15.129:0/1663646672 wait complete. 2024-04-25T01:01:58.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7fba6f59e640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fba700c5c90 con 0x7fba700c3150 2024-04-25T01:01:58.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/115811633 shutdown_connections 2024-04-25T01:01:58.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.251+0000 7fba6f59e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fba70153e80 con 0x7fba700a7030 2024-04-25T01:01:58.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6dd9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba700a7030 0x7fba700c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6ed9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fba700c3150 0x7fba700c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fc4a7fff640 1 --2- 172.21.15.129:0/1763226110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 0x7fc4a00d3b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc49c010cb0 tx=0x7fc49c032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32d12c45ba4bf48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6e59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fba701506f0 0x7fba7006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44742/0 (socket says 172.21.15.129:44742) 2024-04-25T01:01:58.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f4eab59e640 1 Processor -- start 2024-04-25T01:01:58.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6e59c640 1 -- 172.21.15.129:0/1827924462 learned_addr learned my addr 172.21.15.129:0/1827924462 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:58.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6d59a640 1 -- 172.21.15.129:0/1827924462 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3932914262 0 0) 0x7fba70153270 con 0x7fba701506f0 2024-04-25T01:01:58.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/115811633 wait complete. 2024-04-25T01:01:58.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fc4a6ffd640 1 -- 172.21.15.129:0/1763226110 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c039070 con 0x7fc4a000bd50 2024-04-25T01:01:58.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fc4a6ffd640 1 -- 172.21.15.129:0/1763226110 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49c033940 con 0x7fc4a000bd50 2024-04-25T01:01:58.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fc4a6ffd640 1 -- 172.21.15.129:0/1763226110 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c033d10 con 0x7fc4a000bd50 2024-04-25T01:01:58.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7ff84e422640 1 Processor -- start 2024-04-25T01:01:58.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7ffa0b509640 1 Processor -- start 2024-04-25T01:01:58.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6dd9b640 1 -- 172.21.15.129:0/1827924462 >> v2:172.21.15.129:3300/0 conn(0x7fba700c3150 msgr2=0x7fba700c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7fba6dd9b640 1 --2- 172.21.15.129:0/1827924462 >> v2:172.21.15.129:3300/0 conn(0x7fba700c3150 0x7fba700c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7fba6dd9b640 1 -- 172.21.15.129:0/1827924462 >> v1:172.21.15.129:6789/0 conn(0x7fba701506f0 legacy=0x7fba7006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7fba6dd9b640 1 -- 172.21.15.129:0/1827924462 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba70153550 con 0x7fba700a7030 2024-04-25T01:01:58.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7ff84e422640 1 -- start start 2024-04-25T01:01:58.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.252+0000 7f4eab59e640 1 -- start start 2024-04-25T01:01:58.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4eab59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac0aa720 0x7f4eac15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4eab59e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4eac0c6be0 0x7f4eac158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4eab59e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac156cd0 0x7f4eac0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4eab59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac159cd0 con 0x7f4eac0aa720 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4eab59e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac06e200 con 0x7f4eac0c6be0 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4eab59e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac159e50 con 0x7f4eac156cd0 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7fc4ad32e640 1 -- 172.21.15.129:0/1763226110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 msgr2=0x7fc4a00d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/1763226110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 0x7fc4a00d3b40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc49c010cb0 tx=0x7fc49c032c30 comp rx=0 tx=0).stop 2024-04-25T01:01:58.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f1fcdfe4640 1 Processor -- start 2024-04-25T01:01:58.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f4ea9d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4eac0c6be0 0x7f4eac158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fba6dd9b640 1 --2- 172.21.15.129:0/1827924462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba700a7030 0x7fba700c5750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba640097a0 tx=0x7fba6402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6394fbde09cb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4ea9d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4eac0c6be0 0x7f4eac158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54604/0 (socket says 172.21.15.129:54604) 2024-04-25T01:01:58.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4ea9d9b640 1 -- 172.21.15.129:0/1707680349 learned_addr learned my addr 172.21.15.129:0/1707680349 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f1fcdfe4640 1 -- start start 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fc4ad32e640 1 -- 172.21.15.129:0/1763226110 shutdown_connections 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/1763226110 >> v2:172.21.15.171:3300/0 conn(0x7fc4a004b110 0x7fc4a0046b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/1763226110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a000bd50 0x7fc4a00d3b40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fc4ad32e640 1 -- 172.21.15.129:0/1763226110 >> 172.21.15.129:0/1763226110 conn(0x7fc4a0048250 msgr2=0x7fc4a0048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fc4ad32e640 1 -- 172.21.15.129:0/1763226110 shutdown_connections 2024-04-25T01:01:58.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7fc4ad32e640 1 -- 172.21.15.129:0/1763226110 wait complete. 2024-04-25T01:01:58.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7ffa0b509640 1 -- start start 2024-04-25T01:01:58.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84e422640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff840007410 0x7ff8401783d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ffa0b509640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9fc00b100 0x7ff9fc041370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f13d359e640 1 Processor -- start 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.253+0000 7f13d359e640 1 -- start start 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f13d359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13cc006250 0x7f13cc0dbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f13d359e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f13cc0e4520 0x7f13cc0300f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f13d359e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc180c90 0x7f13cc030630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f13d359e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc00f4c0 con 0x7f13cc006250 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f13d359e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc0e4900 con 0x7f13cc180c90 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f13d359e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f13cc00f640 con 0x7f13cc0e4520 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcdfe4640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc004bf40 0x7f1fc00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ffa0b509640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcdfe4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc00d5970 0x7f1fc016eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcdfe4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 0x7f1fc016f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff847fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff840007410 0x7ff8401783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcdfe4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0048bf0 con 0x7f1fc00d5970 2024-04-25T01:01:58.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcdfe4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0048a70 con 0x7f1fc00d8240 2024-04-25T01:01:58.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff847fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff840007410 0x7ff8401783d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38590/0 (socket says 172.21.15.129:38590) 2024-04-25T01:01:58.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcdfe4640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00d5d50 con 0x7f1fc004bf40 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac156cd0 0x7f4eac0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 -- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4eac0c6be0 msgr2=0x7f4eac158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4eac0c6be0 0x7f4eac158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 -- 172.21.15.129:0/1707680349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac0aa720 msgr2=0x7f4eac15ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac0aa720 0x7f4eac15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff847fff640 1 -- 172.21.15.129:0/1077701801 learned_addr learned my addr 172.21.15.129:0/1077701801 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 -- 172.21.15.129:0/1707680349 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac1f0330 con 0x7f4eac156cd0 2024-04-25T01:01:58.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.254+0000 7f4eaad9d640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac156cd0 0x7f4eac0a3a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c7b0 tx=0x7f4ea400cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4017070 con 0x7f4eac156cd0 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea4003d80 con 0x7f4eac156cd0 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea400ce60 con 0x7f4eac156cd0 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8400474d0 0x7ff8400d8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f13d2d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc180c90 0x7f13cc030630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f13d2d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc180c90 0x7f13cc030630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54608/0 (socket says 172.21.15.129:54608) 2024-04-25T01:01:58.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f13d2d9d640 1 -- 172.21.15.129:0/2605832633 learned_addr learned my addr 172.21.15.129:0/2605832633 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f13d1d9b640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f13cc0e4520 0x7f13cc0300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7fba6d59a640 1 -- 172.21.15.129:0/1827924462 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64037070 con 0x7fba700a7030 2024-04-25T01:01:58.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7fba6d59a640 1 -- 172.21.15.129:0/1827924462 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6402ae40 con 0x7fba700a7030 2024-04-25T01:01:58.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7fba6d59a640 1 -- 172.21.15.129:0/1827924462 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64032660 con 0x7fba700a7030 2024-04-25T01:01:58.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400dbd10 0x7ff840044980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f4eab59e640 1 -- 172.21.15.129:0/1707680349 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac0a5980 con 0x7f4eac156cd0 2024-04-25T01:01:58.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f4eab59e640 1 -- 172.21.15.129:0/1707680349 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac0a3230 con 0x7f4eac156cd0 2024-04-25T01:01:58.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f13d259c640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13cc006250 0x7f13cc0dbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7fba94ed4640 1 Processor -- start 2024-04-25T01:01:58.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff8477fe640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8400474d0 0x7ff8400d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84c9bf640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400dbd10 0x7ff840044980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400dc0f0 con 0x7ff840007410 2024-04-25T01:01:58.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff840047db0 con 0x7ff8400474d0 2024-04-25T01:01:58.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400dfc20 con 0x7ff8400dbd10 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7f1fcd7e3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 0x7f1fc016f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 0x7f1fc016f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54632/0 (socket says 172.21.15.129:54632) 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 -- 172.21.15.129:0/953544493 learned_addr learned my addr 172.21.15.129:0/953544493 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f13d259c640 1 -- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f13cc0e4520 msgr2=0x7f13cc0300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f13d259c640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f13cc0e4520 0x7f13cc0300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f13d259c640 1 -- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc180c90 msgr2=0x7f13cc030630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f13d259c640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc180c90 0x7f13cc030630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f13d259c640 1 -- 172.21.15.129:0/2605832633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc034880 con 0x7f13cc006250 2024-04-25T01:01:58.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7fc4ad32e640 1 Processor -- start 2024-04-25T01:01:58.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7fc4ad32e640 1 -- start start 2024-04-25T01:01:58.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ff8477fe640 1 -- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400dbd10 msgr2=0x7ff840044980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ff8477fe640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400dbd10 0x7ff840044980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.255+0000 7fba94ed4640 1 -- start start 2024-04-25T01:01:58.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ff8477fe640 1 -- 172.21.15.129:0/1077701801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff840007410 msgr2=0x7ff8401783d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ff8477fe640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff840007410 0x7ff8401783d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ff8477fe640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff840041f30 con 0x7ff8400474d0 2024-04-25T01:01:58.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 -- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc004bf40 msgr2=0x7f1fc00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc004bf40 0x7f1fc00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 -- 172.21.15.129:0/953544493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc00d5970 msgr2=0x7f1fc016eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc00d5970 0x7f1fc016eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7f1fcd7e3640 1 -- 172.21.15.129:0/953544493 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00c6e40 con 0x7f1fc00d8240 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7fba6f59e640 1 -- 172.21.15.129:0/1827924462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba700a7030 msgr2=0x7fba700c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7fba6f59e640 1 --2- 172.21.15.129:0/1827924462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba700a7030 0x7fba700c5750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba640097a0 tx=0x7fba6402a0f0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4ad32e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 0x7fc4a00403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4ad32e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a004b110 0x7fc4a0041f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4ad32e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a00d6710 0x7fc4a0179a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4ad32e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a00da220 con 0x7fc4a00d6710 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4ad32e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a00d9f40 con 0x7fc4a000bd50 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4ad32e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a00da500 con 0x7fc4a004b110 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f1fcd7e3640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 0x7f1fc016f0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fc802d700 tx=0x7f1fc8073750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f13d259c640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13cc006250 0x7f13cc0dbd70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f13c400e660 tx=0x7f13c400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba6f59e640 1 -- 172.21.15.129:0/1827924462 shutdown_connections 2024-04-25T01:01:58.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba6f59e640 1 --2- 172.21.15.129:0/1827924462 >> v2:172.21.15.129:3300/0 conn(0x7fba700c3150 0x7fba700c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba6f59e640 1 --2- 172.21.15.129:0/1827924462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba700a7030 0x7fba700c5750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba6f59e640 1 -- 172.21.15.129:0/1827924462 >> 172.21.15.129:0/1827924462 conn(0x7fba700a95e0 msgr2=0x7fba700a99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7ff8477fe640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8400474d0 0x7ff8400d8690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff83c031000 tx=0x7ff83c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa0b509640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff9fc0d6fd0 0x7ff9fc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa0b509640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc046210 con 0x7ff9fc0d3be0 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa08aa4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa08aa4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38592/0 (socket says 172.21.15.129:38592) 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa08aa4640 1 -- 172.21.15.129:0/175283698 learned_addr learned my addr 172.21.15.129:0/175283698 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc0d6600 con 0x7ff9fc00b100 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.256+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff9fc046390 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7ffa092a5640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9fc00b100 0x7ff9fc041370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba6f59e640 1 -- 172.21.15.129:0/1827924462 shutdown_connections 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8072590 con 0x7f1fc00d8240 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba94ed4640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900abe00 0x7fba900c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba94ed4640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fba900c2200 0x7fba90155080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba94ed4640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba900c4dd0 con 0x7fba900aba20 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba94ed4640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fba900c5670 con 0x7fba900c2200 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba8e59c640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fba900aba20 0x7fba900a5f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54064/0 (socket says 172.21.15.129:54064) 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba8e59c640 1 -- 172.21.15.129:0/67221776 learned_addr learned my addr 172.21.15.129:0/67221776 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:58.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba94ed4640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fba900c50b0 con 0x7fba900abe00 2024-04-25T01:01:58.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc808f030 con 0x7f1fc00d8240 2024-04-25T01:01:58.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 0x7fc4a00403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 0x7fc4a00403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54658/0 (socket says 172.21.15.129:54658) 2024-04-25T01:01:58.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 -- 172.21.15.129:0/3088155489 learned_addr learned my addr 172.21.15.129:0/3088155489 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc00d7c20 con 0x7f1fc00d8240 2024-04-25T01:01:58.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc00d6070 con 0x7f1fc00d8240 2024-04-25T01:01:58.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a77fe640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a004b110 0x7fc4a0041f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8072a90 con 0x7f1fc00d8240 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 -- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a004b110 msgr2=0x7fc4a0041f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a004b110 0x7fc4a0041f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 -- 172.21.15.129:0/3088155489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a00d6710 msgr2=0x7fc4a0179a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a00d6710 0x7fc4a0179a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fc4a7fff640 1 -- 172.21.15.129:0/3088155489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a003f300 con 0x7fc4a000bd50 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fc4a7fff640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 0x7fc4a00403f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc49c010cb0 tx=0x7fc49c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7ffa09aa6640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff9fc0d6fd0 0x7ff9fc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c042020 con 0x7ff8400474d0 2024-04-25T01:01:58.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c03d750 con 0x7ff8400474d0 2024-04-25T01:01:58.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c037040 con 0x7ff8400474d0 2024-04-25T01:01:58.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c019560 con 0x7fc4a000bd50 2024-04-25T01:01:58.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49c03dab0 con 0x7fc4a000bd50 2024-04-25T01:01:58.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff840041110 con 0x7ff8400474d0 2024-04-25T01:01:58.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8400416d0 con 0x7ff8400474d0 2024-04-25T01:01:58.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c046410 con 0x7fc4a000bd50 2024-04-25T01:01:58.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4017070 con 0x7f13cc006250 2024-04-25T01:01:58.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc1801b0 con 0x7f13cc006250 2024-04-25T01:01:58.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc0dc2b0 con 0x7f13cc006250 2024-04-25T01:01:58.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.259+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a0044c40 con 0x7fc4a000bd50 2024-04-25T01:01:58.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.259+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a003f8a0 con 0x7fc4a000bd50 2024-04-25T01:01:58.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/67221776 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900abe00 0x7fba900c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fba8dd9b640 1 -- 172.21.15.129:0/67221776 >> v2:172.21.15.129:3300/0 conn(0x7fba900c2200 msgr2=0x7fba90155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:58.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/67221776 >> v2:172.21.15.129:3300/0 conn(0x7fba900c2200 0x7fba90155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fba8dd9b640 1 -- 172.21.15.129:0/67221776 >> v1:172.21.15.171:6789/0 conn(0x7fba900aba20 legacy=0x7fba900a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7fba8dd9b640 1 -- 172.21.15.129:0/67221776 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba900c5390 con 0x7fba900abe00 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7ffa09aa6640 1 -- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9fc00b100 msgr2=0x7ff9fc041370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7ffa09aa6640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9fc00b100 0x7ff9fc041370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7ffa09aa6640 1 -- 172.21.15.129:0/175283698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 msgr2=0x7ff9fc0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7ffa09aa6640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7ffa09aa6640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc16c4d0 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.259+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c400c730 con 0x7f13cc006250 2024-04-25T01:01:58.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c400cae0 con 0x7f13cc006250 2024-04-25T01:01:58.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/67221776 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900abe00 0x7fba900c1cc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba800097a0 tx=0x7fba8002a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9aad19c34a6ca71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fc8072c30 con 0x7f1fc00d8240 2024-04-25T01:01:58.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7ffa09aa6640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff9fc0d6fd0 0x7ff9fc0cfd70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400b3e0 tx=0x7ff9f400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7fba6f59e640 1 -- 172.21.15.129:0/1827924462 wait complete. 2024-04-25T01:01:58.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.259+0000 7fba6f59e640 1 Processor -- start 2024-04-25T01:01:58.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.259+0000 7fba6f59e640 1 -- start start 2024-04-25T01:01:58.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba700a7030 0x7fba7014bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6f59e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba700c3150 0x7fba7014d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6f59e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 0x7fba7014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6f59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba70150ad0 con 0x7fba700a7030 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6f59e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba70153b10 con 0x7fba700c3150 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6f59e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fba7006b780 con 0x7fba701506f0 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 0x7fba7014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 0x7fba7014da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47762/0 (socket says 172.21.15.129:47762) 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 -- 172.21.15.129:0/3272895728 learned_addr learned my addr 172.21.15.129:0/3272895728 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 -- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba700c3150 msgr2=0x7fba7014d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba700c3150 0x7fba7014d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 -- 172.21.15.129:0/3272895728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba700a7030 msgr2=0x7fba7014bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba700a7030 0x7fba7014bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7f1fc57fa640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1fac06be80 0x7f1fac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc49c044020 con 0x7fc4a000bd50 2024-04-25T01:01:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8400dc360 con 0x7ff8400474d0 2024-04-25T01:01:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1fc8106da0 con 0x7f1fc00d8240 2024-04-25T01:01:58.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.257+0000 7f4eab59e640 1 -- 172.21.15.129:0/1707680349 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac075440 con 0x7f4eac156cd0 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ea401a070 con 0x7f4eac156cd0 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.258+0000 7f4e9affd640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4e8c06bf30 0x7f4e8c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.259+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4ea408a7f0 con 0x7f4eac156cd0 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7f4eaa59c640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4e8c06bf30 0x7f4e8c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7f4eaa59c640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4e8c06bf30 0x7f4e8c06e3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c034000 tx=0x7f4e9c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7f1fccfe2640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1fac06be80 0x7f1fac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa8005290 con 0x7f1fc00d8240 2024-04-25T01:01:58.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff83c045020 con 0x7ff8400474d0 2024-04-25T01:01:58.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.260+0000 7fba6ed9d640 1 -- 172.21.15.129:0/3272895728 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7014e170 con 0x7fba701506f0 2024-04-25T01:01:58.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a0005390 con 0x7fc4a000bd50 2024-04-25T01:01:58.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7fba6ed9d640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 0x7fba7014da20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba68009dd0 tx=0x7fba6800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.261+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b8005290 con 0x7f13cc006250 2024-04-25T01:01:58.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7fc4a4ff9640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc48c06bf00 0x7fc48c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7ff844ff9640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff82c06bf00 0x7ff82c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff83c0abc10 con 0x7ff8400474d0 2024-04-25T01:01:58.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc49c0ab7a0 con 0x7fc4a000bd50 2024-04-25T01:01:58.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7fc4a77fe640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc48c06bf00 0x7fc48c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7f1fccfe2640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1fac06be80 0x7f1fac06e340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4034000 tx=0x7f1fb402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7ff847fff640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff82c06bf00 0x7ff82c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13c401a070 con 0x7f13cc006250 2024-04-25T01:01:58.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.262+0000 7fc4a77fe640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc48c06bf00 0x7fc48c06e3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc494006fd0 tx=0x7fc494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.263+0000 7ff847fff640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff82c06bf00 0x7ff82c06e3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff834000f60 tx=0x7ff834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.263+0000 7f13c2ffd640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f13b406be80 0x7f13b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.263+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f13c408b050 con 0x7f13cc006250 2024-04-25T01:01:58.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.263+0000 7f13d1d9b640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f13b406be80 0x7f13b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.263+0000 7f13d1d9b640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f13b406be80 0x7f13b406e340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00df60 tx=0x7f13c8000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7fba8d59a640 1 -- 172.21.15.129:0/67221776 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80037070 con 0x7fba900abe00 2024-04-25T01:01:58.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680135c0 con 0x7fba701506f0 2024-04-25T01:01:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4015020 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f4011850 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc16c7b0 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4011c40 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc16cd70 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4ea4058ed0 con 0x7f4eac156cd0 2024-04-25T01:01:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba68017070 con 0x7fba701506f0 2024-04-25T01:01:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba680139c0 con 0x7fba701506f0 2024-04-25T01:01:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7fba6f59e640 1 -- 172.21.15.129:0/3272895728 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701438f0 con 0x7fba701506f0 2024-04-25T01:01:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.264+0000 7fba6f59e640 1 -- 172.21.15.129:0/3272895728 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7014ecf0 con 0x7fba701506f0 2024-04-25T01:01:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.265+0000 7fba8d59a640 1 -- 172.21.15.129:0/67221776 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8002aca0 con 0x7fba900abe00 2024-04-25T01:01:58.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.265+0000 7fba8d59a640 1 -- 172.21.15.129:0/67221776 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80032660 con 0x7fba900abe00 2024-04-25T01:01:58.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.265+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9fc0493d0 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba48005290 con 0x7fba701506f0 2024-04-25T01:01:58.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 -- 172.21.15.129:0/67221776 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900abe00 msgr2=0x7fba900c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 --2- 172.21.15.129:0/67221776 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900abe00 0x7fba900c1cc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba800097a0 tx=0x7fba8002a010 comp rx=0 tx=0).stop 2024-04-25T01:01:58.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 -- 172.21.15.129:0/67221776 shutdown_connections 2024-04-25T01:01:58.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 --2- 172.21.15.129:0/67221776 >> v2:172.21.15.129:3300/0 conn(0x7fba900c2200 0x7fba90155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 --2- 172.21.15.129:0/67221776 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900abe00 0x7fba900c1cc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 -- 172.21.15.129:0/67221776 >> 172.21.15.129:0/67221776 conn(0x7fba900a86b0 msgr2=0x7fba900a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9f4018020 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba94ed4640 1 -- 172.21.15.129:0/67221776 shutdown_connections 2024-04-25T01:01:58.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba68022020 con 0x7fba701506f0 2024-04-25T01:01:58.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.266+0000 7ff9f9ffb640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff9e806be30 0x7ff9e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7fba94ed4640 1 -- 172.21.15.129:0/67221776 wait complete. 2024-04-25T01:01:58.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff9f408af90 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7ffa092a5640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff9e806be30 0x7ff9e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7fba5effd640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba5006be60 0x7fba5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8400dc360 con 0x7ff8400474d0 2024-04-25T01:01:58.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba6808f930 con 0x7fba701506f0 2024-04-25T01:01:58.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7fba6e59c640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba5006be60 0x7fba5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.267+0000 7ffa092a5640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff9e806be30 0x7ff9e806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006fd0 tx=0x7ffa00009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.268+0000 7fba6e59c640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba5006be60 0x7fba5006e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba60000c00 tx=0x7fba600079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.268+0000 7fba94ed4640 1 Processor -- start 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.268+0000 7fba94ed4640 1 -- start start 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba94ed4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 0x7fba901f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba94ed4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba900abe00 0x7fba900bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba94ed4640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900c2200 0x7fba900bf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba94ed4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fba9006c3f0 con 0x7fba900abe00 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba94ed4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba9006c270 con 0x7fba900aba20 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba94ed4640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fba900ac280 con 0x7fba900c2200 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 0x7fba901f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 0x7fba901f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54676/0 (socket says 172.21.15.129:54676) 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 -- 172.21.15.129:0/2007845116 learned_addr learned my addr 172.21.15.129:0/2007845116 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:58.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba900abe00 0x7fba900bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8ed9d640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900c2200 0x7fba900bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 -- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900c2200 msgr2=0x7fba900bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900c2200 0x7fba900bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 -- 172.21.15.129:0/2007845116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba900abe00 msgr2=0x7fba900bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba900abe00 0x7fba900bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 -- 172.21.15.129:0/2007845116 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba901f3640 con 0x7fba900aba20 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8ed9d640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900c2200 0x7fba900bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba900abe00 0x7fba900bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.269+0000 7fba8e59c640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 0x7fba901f2b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c5d0 tx=0x7fba8400ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.270+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13c4059150 con 0x7f13cc006250 2024-04-25T01:01:58.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.271+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fc80d54b0 con 0x7f1fc00d8240 2024-04-25T01:01:58.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.272+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9fc0493d0 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.275+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84018070 con 0x7fba900aba20 2024-04-25T01:01:58.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.275+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba840074e0 con 0x7fba900aba20 2024-04-25T01:01:58.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.276+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc49c033910 con 0x7fc4a000bd50 2024-04-25T01:01:58.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.276+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84007690 con 0x7fba900aba20 2024-04-25T01:01:58.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.276+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba900728d0 con 0x7fba900aba20 2024-04-25T01:01:58.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.276+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba900a7580 con 0x7fba900aba20 2024-04-25T01:01:58.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.276+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba6805e010 con 0x7fba701506f0 2024-04-25T01:01:58.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.278+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba90075550 con 0x7fba900aba20 2024-04-25T01:01:58.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.279+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba84004a20 con 0x7fba900aba20 2024-04-25T01:01:58.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.279+0000 7fba7effd640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba7006be30 0x7fba7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:58.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.279+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba8408a500 con 0x7fba900aba20 2024-04-25T01:01:58.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.280+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba7006be30 0x7fba7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:58.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.281+0000 7fba8dd9b640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba7006be30 0x7fba7006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:58.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.284+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba84058510 con 0x7fba900aba20 2024-04-25T01:01:58.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.683+0000 7f4eab59e640 1 -- 172.21.15.129:0/1707680349 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4eac0a5b50 con 0x7f4eac156cd0 2024-04-25T01:01:58.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.684+0000 7f4e9affd640 1 -- 172.21.15.129:0/1707680349 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4eac0a5b50 con 0x7f4eac156cd0 2024-04-25T01:01:58.684 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.686+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f13b8005850 con 0x7f13cc006250 2024-04-25T01:01:58.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.687+0000 7f13c2ffd640 1 -- 172.21.15.129:0/2605832633 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13c405d450 con 0x7f13cc006250 2024-04-25T01:01:58.687 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 -- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4e8c06bf30 msgr2=0x7f4e8c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4e8c06bf30 0x7f4e8c06e3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c034000 tx=0x7f4e9c02c000 comp rx=0 tx=0).stop 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 -- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac156cd0 msgr2=0x7f4eac0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac156cd0 0x7f4eac0a3a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c7b0 tx=0x7f4ea400cc70 comp rx=0 tx=0).stop 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 -- 172.21.15.129:0/1707680349 shutdown_connections 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4e8c06bf30 0x7f4e8c06e3f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4eac156cd0 0x7f4eac0a3a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4eac0c6be0 0x7f4eac158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 --2- 172.21.15.129:0/1707680349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eac0aa720 0x7f4eac15ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 -- 172.21.15.129:0/1707680349 >> 172.21.15.129:0/1707680349 conn(0x7f4eac06cc80 msgr2=0x7f4eac0a6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 -- 172.21.15.129:0/1707680349 shutdown_connections 2024-04-25T01:01:58.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.688+0000 7f4e98ff9640 1 -- 172.21.15.129:0/1707680349 wait complete. 2024-04-25T01:01:58.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.689+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff9fc046e20 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.689+0000 7ff9f9ffb640 1 -- 172.21.15.129:0/175283698 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff9f401f020 con 0x7ff9fc0d6fd0 2024-04-25T01:01:58.691 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:01:58.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.690+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff8400053c0 con 0x7ff8400474d0 2024-04-25T01:01:58.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.690+0000 7ff844ff9640 1 -- 172.21.15.129:0/1077701801 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff83c079ca0 con 0x7ff8400474d0 2024-04-25T01:01:58.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.690+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f13b406be80 msgr2=0x7f13b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.690+0000 7f13d359e640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f13b406be80 0x7f13b406e340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00df60 tx=0x7f13c8000ed0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.692 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13cc006250 msgr2=0x7f13cc0dbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13cc006250 0x7f13cc0dbd70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f13c400e660 tx=0x7f13c400eb20 comp rx=0 tx=0).stop 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 shutdown_connections 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f13b406be80 0x7f13b406e340 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f13cc0e4520 0x7f13cc0300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13cc180c90 0x7f13cc030630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 --2- 172.21.15.129:0/2605832633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13cc006250 0x7f13cc0dbd70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 >> 172.21.15.129:0/2605832633 conn(0x7f13cc04fc00 msgr2=0x7f13cc0dcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.691+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 shutdown_connections 2024-04-25T01:01:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.692+0000 7f13d359e640 1 -- 172.21.15.129:0/2605832633 wait complete. 2024-04-25T01:01:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.692+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc4a0046bb0 con 0x7fc4a000bd50 2024-04-25T01:01:58.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.693+0000 7fc4a4ff9640 1 -- 172.21.15.129:0/3088155489 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc49c079830 con 0x7fc4a000bd50 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.693+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff9e806be30 msgr2=0x7ff9e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff9e806be30 0x7ff9e806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006fd0 tx=0x7ffa00009060 comp rx=0 tx=0).stop 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff9fc0d6fd0 msgr2=0x7ff9fc0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff9fc0d6fd0 0x7ff9fc0cfd70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400b3e0 tx=0x7ff9f400b8a0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff82c06bf00 msgr2=0x7ff82c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff82c06bf00 0x7ff82c06e3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff834000f60 tx=0x7ff834009040 comp rx=0 tx=0).stop 2024-04-25T01:01:58.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8400474d0 msgr2=0x7ff8400d8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8400474d0 0x7ff8400d8690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff83c031000 tx=0x7ff83c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 shutdown_connections 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff82c06bf00 0x7ff82c06e3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff8400dbd10 0x7ff840044980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8400474d0 0x7ff8400d8690 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 --2- 172.21.15.129:0/1077701801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff840007410 0x7ff8401783d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 >> 172.21.15.129:0/1077701801 conn(0x7ff840048280 msgr2=0x7ff84004a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 shutdown_connections 2024-04-25T01:01:58.696 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff9e806be30 0x7ff9e806e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff9fc0d6fd0 0x7ff9fc0cfd70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9fc0d3be0 0x7ff9fc0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 --2- 172.21.15.129:0/175283698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9fc00b100 0x7ff9fc041370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 >> 172.21.15.129:0/175283698 conn(0x7ff9fc005ec0 msgr2=0x7ff9fc0069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 shutdown_connections 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.694+0000 7ffa0b509640 1 -- 172.21.15.129:0/175283698 wait complete. 2024-04-25T01:01:58.697 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:01:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.695+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 shutdown_connections 2024-04-25T01:01:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.695+0000 7ff84e422640 1 -- 172.21.15.129:0/1077701801 wait complete. 2024-04-25T01:01:58.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1fa8005580 con 0x7f1fc00d8240 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc48c06bf00 msgr2=0x7fc48c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc48c06bf00 0x7fc48c06e3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc494006fd0 tx=0x7fc494008040 comp rx=0 tx=0).stop 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 msgr2=0x7fc4a00403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 0x7fc4a00403f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc49c010cb0 tx=0x7fc49c004730 comp rx=0 tx=0).stop 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 shutdown_connections 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc48c06bf00 0x7fc48c06e3c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4a004b110 0x7fc4a0041f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4a00d6710 0x7fc4a0179a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 --2- 172.21.15.129:0/3088155489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4a000bd50 0x7fc4a00403f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.698+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 >> 172.21.15.129:0/3088155489 conn(0x7fc4a0048250 msgr2=0x7fc4a00049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.699+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 shutdown_connections 2024-04-25T01:01:58.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.699+0000 7fc4ad32e640 1 -- 172.21.15.129:0/3088155489 wait complete. 2024-04-25T01:01:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.699+0000 7f1fc57fa640 1 -- 172.21.15.129:0/953544493 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fc80d4db0 con 0x7f1fc00d8240 2024-04-25T01:01:58.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.699+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fba48005850 con 0x7fba701506f0 2024-04-25T01:01:58.701 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:01:58.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.700+0000 7fba5effd640 1 -- 172.21.15.129:0/3272895728 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba6805d910 con 0x7fba701506f0 2024-04-25T01:01:58.702 INFO:teuthology.orchestra.run.smithi129.stdout:55834574856 2024-04-25T01:01:58.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.702+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fba900c4b10 con 0x7fba900aba20 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1fac06be80 msgr2=0x7f1fac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1fac06be80 0x7f1fac06e340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4034000 tx=0x7f1fb402c000 comp rx=0 tx=0).stop 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 msgr2=0x7f1fc016f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 0x7f1fc016f0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fc802d700 tx=0x7f1fc8073750 comp rx=0 tx=0).stop 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 shutdown_connections 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1fac06be80 0x7f1fac06e340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fc00d8240 0x7f1fc016f0d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fc00d5970 0x7f1fc016eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 --2- 172.21.15.129:0/953544493 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1fc004bf40 0x7f1fc00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 >> 172.21.15.129:0/953544493 conn(0x7f1fc004d9e0 msgr2=0x7f1fc004ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7fba7effd640 1 -- 172.21.15.129:0/2007845116 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba8405c810 con 0x7fba900aba20 2024-04-25T01:01:58.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 shutdown_connections 2024-04-25T01:01:58.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.703+0000 7f1fcdfe4640 1 -- 172.21.15.129:0/953544493 wait complete. 2024-04-25T01:01:58.705 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba5006be60 msgr2=0x7fba5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba5006be60 0x7fba5006e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba60000c00 tx=0x7fba600079c0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 msgr2=0x7fba7014da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 0x7fba7014da20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fba68009dd0 tx=0x7fba6800c490 comp rx=0 tx=0).stop 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 shutdown_connections 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba5006be60 0x7fba5006e320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba701506f0 0x7fba7014da20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba700c3150 0x7fba7014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 --2- 172.21.15.129:0/3272895728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba700a7030 0x7fba7014bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 >> 172.21.15.129:0/3272895728 conn(0x7fba700a95e0 msgr2=0x7fba700c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 shutdown_connections 2024-04-25T01:01:58.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.704+0000 7fba5cff9640 1 -- 172.21.15.129:0/3272895728 wait complete. 2024-04-25T01:01:58.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.708+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba7006be30 msgr2=0x7fba7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.708+0000 7fba94ed4640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba7006be30 0x7fba7006e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba80034000 tx=0x7fba8002e000 comp rx=0 tx=0).stop 2024-04-25T01:01:58.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.708+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 msgr2=0x7fba901f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:58.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.708+0000 7fba94ed4640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 0x7fba901f2b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c5d0 tx=0x7fba8400ca90 comp rx=0 tx=0).stop 2024-04-25T01:01:58.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.709+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 shutdown_connections 2024-04-25T01:01:58.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.709+0000 7fba94ed4640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fba7006be30 0x7fba7006e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.709+0000 7fba94ed4640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba900c2200 0x7fba900bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.709+0000 7fba94ed4640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fba900abe00 0x7fba900bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.709+0000 7fba94ed4640 1 --2- 172.21.15.129:0/2007845116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba900aba20 0x7fba901f2b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:58.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.709+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 >> 172.21.15.129:0/2007845116 conn(0x7fba900a86b0 msgr2=0x7fba90155080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:58.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.710+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 shutdown_connections 2024-04-25T01:01:58.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:58.710+0000 7fba94ed4640 1 -- 172.21.15.129:0/2007845116 wait complete. 2024-04-25T01:01:58.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-25T01:01:58.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-04-25T01:01:58.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-25T01:01:58.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-04-25T01:01:58.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-25T01:01:58.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-04-25T01:01:58.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-04-25T01:01:58.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-04-25T01:01:59.717 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T01:01:59.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T01:01:59.719 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T01:01:59.722 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T01:01:59.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T01:01:59.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T01:01:59.729 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T01:01:59.731 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T01:01:59.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.867+0000 7f01b01e7640 1 Processor -- start 2024-04-25T01:01:59.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.867+0000 7f01b01e7640 1 -- start start 2024-04-25T01:01:59.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.868+0000 7f01b01e7640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f01a8152070 0x7f01a80a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.868+0000 7f01b01e7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f01a806c270 0x7f01a80c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.868+0000 7f01b01e7640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a80c65f0 con 0x7f01a80ac090 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.868+0000 7f01b01e7640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f01a806c680 con 0x7f01a806c270 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.868+0000 7f01b01e7640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f01a80c5a50 con 0x7f01a8152070 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.869+0000 7f01ae784640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f01a806c270 0x7f01a80c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.869+0000 7f01ae784640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f01a806c270 0x7f01a80c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38688/0 (socket says 172.21.15.129:38688) 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.869+0000 7f01ae784640 1 -- 172.21.15.129:0/2704568759 learned_addr learned my addr 172.21.15.129:0/2704568759 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.870+0000 7f01ae784640 1 -- 172.21.15.129:0/2704568759 >> v2:172.21.15.171:3300/0 conn(0x7f01a8152070 msgr2=0x7f01a80a5f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.870+0000 7f01ae784640 1 --2- 172.21.15.129:0/2704568759 >> v2:172.21.15.171:3300/0 conn(0x7f01a8152070 0x7f01a80a5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.870+0000 7f01ae784640 1 -- 172.21.15.129:0/2704568759 >> v1:172.21.15.171:6789/0 conn(0x7f01a80ac090 legacy=0x7f01a80c5ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.870+0000 7f01ae784640 1 -- 172.21.15.129:0/2704568759 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a80c1e60 con 0x7f01a806c270 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.871+0000 7f01ae784640 1 --2- 172.21.15.129:0/2704568759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a806c270 0x7f01a80c4c80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01a80c4480 tx=0x7f01a002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2f7d2508c56f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.872+0000 7f01acf81640 1 -- 172.21.15.129:0/2704568759 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a00090d0 con 0x7f01a806c270 2024-04-25T01:01:59.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.872+0000 7f01acf81640 1 -- 172.21.15.129:0/2704568759 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a00313f0 con 0x7f01a806c270 2024-04-25T01:01:59.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.873+0000 7f01b01e7640 1 -- 172.21.15.129:0/2704568759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a806c270 msgr2=0x7f01a80c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.873+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2704568759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a806c270 0x7f01a80c4c80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f01a80c4480 tx=0x7f01a002a8e0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.874+0000 7f01b01e7640 1 -- 172.21.15.129:0/2704568759 shutdown_connections 2024-04-25T01:01:59.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.874+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2704568759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a806c270 0x7f01a80c4c80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.874+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2704568759 >> v2:172.21.15.171:3300/0 conn(0x7f01a8152070 0x7f01a80a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.874+0000 7f01b01e7640 1 -- 172.21.15.129:0/2704568759 >> 172.21.15.129:0/2704568759 conn(0x7f01a80a86b0 msgr2=0x7f01a80a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.874+0000 7f01b01e7640 1 -- 172.21.15.129:0/2704568759 shutdown_connections 2024-04-25T01:01:59.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.874+0000 7f01b01e7640 1 -- 172.21.15.129:0/2704568759 wait complete. 2024-04-25T01:01:59.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.876+0000 7f01b01e7640 1 Processor -- start 2024-04-25T01:01:59.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.876+0000 7f01b01e7640 1 -- start start 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.876+0000 7f0625bfb640 1 Processor -- start 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.876+0000 7f0625bfb640 1 -- start start 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f0625bfb640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f062006da00 0x7f0620165340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f0625bfb640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f062008b5f0 0x7f0620165880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f0625bfb640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062008bbb0 con 0x7f06200c2d90 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f0625bfb640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0620161de0 con 0x7f062006da00 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f0625bfb640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f0620162030 con 0x7f062008b5f0 2024-04-25T01:01:59.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f061f7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f06200c2d90 0x7f062006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44748/0 (socket says 172.21.15.129:44748) 2024-04-25T01:01:59.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f061f7fe640 1 -- 172.21.15.129:0/3437480940 learned_addr learned my addr 172.21.15.129:0/3437480940 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:59.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.877+0000 7f061effd640 1 --2- 172.21.15.129:0/3437480940 >> v2:172.21.15.129:3300/0 conn(0x7f062006da00 0x7f0620165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f061ffff640 1 --2- 172.21.15.129:0/3437480940 >> v2:172.21.15.171:3300/0 conn(0x7f062008b5f0 0x7f0620165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f01b01e7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a80ac090 0x7f01a8073190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f01b01e7640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f01a8152070 0x7f01a8154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f061ffff640 1 -- 172.21.15.129:0/3437480940 >> v2:172.21.15.129:3300/0 conn(0x7f062006da00 msgr2=0x7f0620165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f061ffff640 1 --2- 172.21.15.129:0/3437480940 >> v2:172.21.15.129:3300/0 conn(0x7f062006da00 0x7f0620165340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f01b01e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 0x7f01a81f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f061ffff640 1 -- 172.21.15.129:0/3437480940 >> v1:172.21.15.129:6789/0 conn(0x7f06200c2d90 legacy=0x7f062006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f061ffff640 1 -- 172.21.15.129:0/3437480940 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620162310 con 0x7f062008b5f0 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f01b01e7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8152450 con 0x7f01a80bbf70 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f01b01e7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01a80c1440 con 0x7f01a80ac090 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.878+0000 7f01b01e7640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f01a806ea20 con 0x7f01a8152070 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f01ad782640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f01a8152070 0x7f01a8154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f01adf83640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a80ac090 0x7f01a8073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f01ae784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 0x7f01a81f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f01ae784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 0x7f01a81f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38702/0 (socket says 172.21.15.129:38702) 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.879+0000 7f01ae784640 1 -- 172.21.15.129:0/2595741528 learned_addr learned my addr 172.21.15.129:0/2595741528 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f01ae784640 1 -- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f01a8152070 msgr2=0x7f01a8154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f01ae784640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f01a8152070 0x7f01a8154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f01ae784640 1 -- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a80ac090 msgr2=0x7f01a8073190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f01ae784640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a80ac090 0x7f01a8073190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f01ae784640 1 -- 172.21.15.129:0/2595741528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a80736d0 con 0x7f01a80bbf70 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f01ae784640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 0x7f01a81f78c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b290 tx=0x7f01a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f061ffff640 1 --2- 172.21.15.129:0/3437480940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062008b5f0 0x7f0620165880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0620166470 tx=0x7f0614009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3781f0f58a1b4db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f3595eb3640 1 Processor -- start 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.880+0000 7f3595eb3640 1 -- start start 2024-04-25T01:01:59.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f3595eb3640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f35900ac600 0x7f359008b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f3595eb3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f359006e540 0x7f359006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f3595eb3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f359006dc00 0x7f359006c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f3595eb3640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f35900c64b0 con 0x7f359006e540 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f3595eb3640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f35900c4f60 con 0x7f35900ac600 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f3595eb3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f35900c5240 con 0x7f359006dc00 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f358effd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f359006e540 0x7f359006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f358effd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f359006e540 0x7f359006d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38718/0 (socket says 172.21.15.129:38718) 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358ffff640 1 --2- 172.21.15.129:0/63774817 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f359006dc00 0x7f359006c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.881+0000 7f358effd640 1 -- 172.21.15.129:0/63774817 learned_addr learned my addr 172.21.15.129:0/63774817 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358f7fe640 1 --2- 172.21.15.129:0/63774817 >> v2:172.21.15.171:3300/0 conn(0x7f35900ac600 0x7f359008b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f061e7fc640 1 -- 172.21.15.129:0/3437480940 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061402a990 con 0x7f062008b5f0 2024-04-25T01:01:59.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f061e7fc640 1 -- 172.21.15.129:0/3437480940 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061402ab30 con 0x7f062008b5f0 2024-04-25T01:01:59.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358effd640 1 -- 172.21.15.129:0/63774817 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f359006dc00 msgr2=0x7f359006c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358effd640 1 --2- 172.21.15.129:0/63774817 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f359006dc00 0x7f359006c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.883+0000 7f0625bfb640 1 -- 172.21.15.129:0/3437480940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062008b5f0 msgr2=0x7f0620165880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.883+0000 7f0625bfb640 1 --2- 172.21.15.129:0/3437480940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062008b5f0 0x7f0620165880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0620166470 tx=0x7f0614009290 comp rx=0 tx=0).stop 2024-04-25T01:01:59.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358effd640 1 -- 172.21.15.129:0/63774817 >> v2:172.21.15.171:3300/0 conn(0x7f35900ac600 msgr2=0x7f359008b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358effd640 1 --2- 172.21.15.129:0/63774817 >> v2:172.21.15.171:3300/0 conn(0x7f35900ac600 0x7f359008b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f0625bfb640 1 -- 172.21.15.129:0/3437480940 shutdown_connections 2024-04-25T01:01:59.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f0625bfb640 1 --2- 172.21.15.129:0/3437480940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062008b5f0 0x7f0620165880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f0625bfb640 1 --2- 172.21.15.129:0/3437480940 >> v2:172.21.15.129:3300/0 conn(0x7f062006da00 0x7f0620165340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f0625bfb640 1 -- 172.21.15.129:0/3437480940 >> 172.21.15.129:0/3437480940 conn(0x7f06200c67b0 msgr2=0x7f06200c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f0625bfb640 1 -- 172.21.15.129:0/3437480940 shutdown_connections 2024-04-25T01:01:59.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.882+0000 7f358effd640 1 -- 172.21.15.129:0/63774817 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35900c5520 con 0x7f359006e540 2024-04-25T01:01:59.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f358effd640 1 --2- 172.21.15.129:0/63774817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006e540 0x7f359006d6c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f359006f210 tx=0x7f357802e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f1c3c33b12ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.885+0000 7f358e7fc640 1 -- 172.21.15.129:0/63774817 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357803a070 con 0x7f359006e540 2024-04-25T01:01:59.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.885+0000 7f358e7fc640 1 -- 172.21.15.129:0/63774817 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357800b650 con 0x7f359006e540 2024-04-25T01:01:59.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 -- 172.21.15.129:0/63774817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006e540 msgr2=0x7f359006d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f358e7fc640 1 -- 172.21.15.129:0/63774817 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357803a070 con 0x7f359006e540 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 --2- 172.21.15.129:0/63774817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006e540 0x7f359006d6c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f359006f210 tx=0x7f357802e480 comp rx=0 tx=0).stop 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 -- 172.21.15.129:0/63774817 shutdown_connections 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 --2- 172.21.15.129:0/63774817 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f359006dc00 0x7f359006c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 --2- 172.21.15.129:0/63774817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006e540 0x7f359006d6c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 --2- 172.21.15.129:0/63774817 >> v2:172.21.15.171:3300/0 conn(0x7f35900ac600 0x7f359008b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f3595eb3640 1 -- 172.21.15.129:0/63774817 >> 172.21.15.129:0/63774817 conn(0x7f35900a86b0 msgr2=0x7f35900a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f3595eb3640 1 -- 172.21.15.129:0/63774817 shutdown_connections 2024-04-25T01:01:59.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.884+0000 7f0625bfb640 1 -- 172.21.15.129:0/3437480940 wait complete. 2024-04-25T01:01:59.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a003f070 con 0x7f01a80bbf70 2024-04-25T01:01:59.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f0625bfb640 1 Processor -- start 2024-04-25T01:01:59.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.886+0000 7f0625bfb640 1 -- start start 2024-04-25T01:01:59.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a8072200 con 0x7f01a80bbf70 2024-04-25T01:01:59.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a8072c70 con 0x7f01a80bbf70 2024-04-25T01:01:59.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a00313f0 con 0x7f01a80bbf70 2024-04-25T01:01:59.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f0625bfb640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 0x7f06200c0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f0625bfb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06200c2d90 0x7f06200bba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f0625bfb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0620165880 0x7f06200bf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f0625bfb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f062006b780 con 0x7f06200c2d90 2024-04-25T01:01:59.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f0625bfb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f062006e4f0 con 0x7f0620165880 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f0625bfb640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f062006b900 con 0x7f062006da00 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 0x7f06200c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f061f7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 0x7f06200c0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47794/0 (socket says 172.21.15.129:47794) 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f061f7fe640 1 -- 172.21.15.129:0/2401532095 learned_addr learned my addr 172.21.15.129:0/2401532095 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061ffff640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0620165880 0x7f06200bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061f7fe640 1 -- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0620165880 msgr2=0x7f06200bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061f7fe640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0620165880 0x7f06200bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061f7fe640 1 -- 172.21.15.129:0/2401532095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06200c2d90 msgr2=0x7f06200bba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061effd640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06200c2d90 0x7f06200bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061f7fe640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06200c2d90 0x7f06200bba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061f7fe640 1 -- 172.21.15.129:0/2401532095 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620072df0 con 0x7f062006da00 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061effd640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06200c2d90 0x7f06200bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.888+0000 7f061f7fe640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 0x7f06200c0820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f061000c600 tx=0x7f061000cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.889+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610018070 con 0x7f062006da00 2024-04-25T01:01:59.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.889+0000 7f0625bfb640 1 -- 172.21.15.129:0/2401532095 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06200730d0 con 0x7f062006da00 2024-04-25T01:01:59.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.889+0000 7f0625bfb640 1 -- 172.21.15.129:0/2401532095 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06200ab1a0 con 0x7f062006da00 2024-04-25T01:01:59.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.890+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06100074e0 con 0x7f062006da00 2024-04-25T01:01:59.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.890+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610007690 con 0x7f062006da00 2024-04-25T01:01:59.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.891+0000 7f0625bfb640 1 -- 172.21.15.129:0/2401532095 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c005290 con 0x7f062006da00 2024-04-25T01:01:59.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.894+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f061001b070 con 0x7f062006da00 2024-04-25T01:01:59.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.894+0000 7f0607fff640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f060806c070 0x7f060806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.894+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f061008aa10 con 0x7f062006da00 2024-04-25T01:01:59.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.895+0000 7f061effd640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f060806c070 0x7f060806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.887+0000 7f3595eb3640 1 -- 172.21.15.129:0/63774817 wait complete. 2024-04-25T01:01:59.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.890+0000 7f3595eb3640 1 Processor -- start 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.891+0000 7f3595eb3640 1 -- start start 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f3595eb3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006dc00 0x7f35901ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f3595eb3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f35900ac600 0x7f3590072450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f3595eb3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 0x7f359008a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f3595eb3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f359006c7b0 con 0x7f359006dc00 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f3595eb3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35900c6140 con 0x7f35901eeb60 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f3595eb3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f359006c930 con 0x7f35900ac600 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f358ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 0x7f359008a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f358ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 0x7f359008a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54736/0 (socket says 172.21.15.129:54736) 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f358ffff640 1 -- 172.21.15.129:0/3368843012 learned_addr learned my addr 172.21.15.129:0/3368843012 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f358f7fe640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006dc00 0x7f35901ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.897+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0610022020 con 0x7f062006da00 2024-04-25T01:01:59.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.897+0000 7f061effd640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f060806c070 0x7f060806e530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0620160650 tx=0x7f061800a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.890+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a00090d0 con 0x7f01a80bbf70 2024-04-25T01:01:59.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.891+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0190005290 con 0x7f01a80bbf70 2024-04-25T01:01:59.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.892+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01a003b410 con 0x7f01a80bbf70 2024-04-25T01:01:59.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f358ffff640 1 -- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f35900ac600 msgr2=0x7f3590072450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:01:59.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f358ffff640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f35900ac600 0x7f3590072450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f358ffff640 1 -- 172.21.15.129:0/3368843012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006dc00 msgr2=0x7f35901ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.902+0000 7f019e7fc640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f018c06bf00 0x7f018c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f01adf83640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f018c06bf00 0x7f018c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01a00ab990 con 0x7f01a80bbf70 2024-04-25T01:01:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f01adf83640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f018c06bf00 0x7f018c06e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f01a815e200 tx=0x7f01a4009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.903+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f01a00b11f0 con 0x7f01a80bbf70 2024-04-25T01:01:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.904+0000 7f358ffff640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006dc00 0x7f35901ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.904+0000 7f358ffff640 1 -- 172.21.15.129:0/3368843012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35901ec670 con 0x7f35901eeb60 2024-04-25T01:01:59.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.904+0000 7f358f7fe640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006dc00 0x7f35901ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:59.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.906+0000 7f358ffff640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 0x7f359008a680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f358000caa0 tx=0x7f358000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.912+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358000ad20 con 0x7f35901eeb60 2024-04-25T01:01:59.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.912+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901ee080 con 0x7f35901eeb60 2024-04-25T01:01:59.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.912+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901edd80 con 0x7f35901eeb60 2024-04-25T01:01:59.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.912+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3580003990 con 0x7f35901eeb60 2024-04-25T01:01:59.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.912+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580003d80 con 0x7f35901eeb60 2024-04-25T01:01:59.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.915+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3580005d10 con 0x7f35901eeb60 2024-04-25T01:01:59.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.915+0000 7f3587fff640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f357006be30 0x7f357006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.915+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f358008b3c0 con 0x7f35901eeb60 2024-04-25T01:01:59.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.916+0000 7f358f7fe640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f357006be30 0x7f357006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.916+0000 7f358f7fe640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f357006be30 0x7f357006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3588006fd0 tx=0x7f3588008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.923+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3574005290 con 0x7f35901eeb60 2024-04-25T01:01:59.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.931+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3580059990 con 0x7f35901eeb60 2024-04-25T01:01:59.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.936+0000 7f798d810640 1 Processor -- start 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.937+0000 7f798d810640 1 -- start start 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f798d810640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7980047240 0x7f79800479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f798d810640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f79800df000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f798d810640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7980048090 con 0x7f798003fb00 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f798d810640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f7980048980 con 0x7f7980047240 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f798d810640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7980047700 con 0x7f79800deb90 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f79877fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f79800df000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f79877fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f79800df000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47806/0 (socket says 172.21.15.129:47806) 2024-04-25T01:01:59.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f79877fe640 1 -- 172.21.15.129:0/1266120845 learned_addr learned my addr 172.21.15.129:0/1266120845 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.938+0000 7f79867fc640 1 --2- 172.21.15.129:0/1266120845 >> v2:172.21.15.171:3300/0 conn(0x7f7980047240 0x7f79800479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.939+0000 7f79877fe640 1 -- 172.21.15.129:0/1266120845 >> v2:172.21.15.171:3300/0 conn(0x7f7980047240 msgr2=0x7f79800479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.939+0000 7f79877fe640 1 --2- 172.21.15.129:0/1266120845 >> v2:172.21.15.171:3300/0 conn(0x7f7980047240 0x7f79800479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.939+0000 7f79877fe640 1 -- 172.21.15.129:0/1266120845 >> v1:172.21.15.129:6789/0 conn(0x7f798003fb00 legacy=0x7f79800dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.939+0000 7f79877fe640 1 -- 172.21.15.129:0/1266120845 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7980048330 con 0x7f79800deb90 2024-04-25T01:01:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.940+0000 7f1815ce1640 1 Processor -- start 2024-04-25T01:01:59.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.940+0000 7f6b00dea640 1 Processor -- start 2024-04-25T01:01:59.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.940+0000 7f6b00dea640 1 -- start start 2024-04-25T01:01:59.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.940+0000 7f1815ce1640 1 -- start start 2024-04-25T01:01:59.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.940+0000 7f79877fe640 1 --2- 172.21.15.129:0/1266120845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f79800df000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f798802d700 tx=0x7f79880a5a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9114fa65ac604a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f7985ffb640 1 -- 172.21.15.129:0/1266120845 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880a9900 con 0x7f79800deb90 2024-04-25T01:01:59.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f7985ffb640 1 -- 172.21.15.129:0/1266120845 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79880a9ae0 con 0x7f79800deb90 2024-04-25T01:01:59.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f7985ffb640 1 -- 172.21.15.129:0/1266120845 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880b1400 con 0x7f79800deb90 2024-04-25T01:01:59.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f1815ce1640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18100c26e0 0x7f181015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f1815ce1640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18100c31f0 con 0x7f1810152e10 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f1815ce1640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1810152330 con 0x7f181014f840 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.941+0000 7f1815ce1640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1810152610 con 0x7f18100c26e0 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6b00dea640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc0a9f90 0x7f6afc06e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6b00dea640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6afc154970 0x7f6afc151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6b00dea640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc0c15a0 con 0x7f6afc0a8a50 2024-04-25T01:01:59.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6b00dea640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f6afc06e5d0 con 0x7f6afc154970 2024-04-25T01:01:59.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6b00dea640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f6afc0aa9c0 con 0x7f6afc0a9f90 2024-04-25T01:01:59.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f180ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18100c26e0 0x7f181015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f180f7fe640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f181014f840 0x7f18100c36c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54068/0 (socket says 172.21.15.129:54068) 2024-04-25T01:01:59.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afbfff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6afc154970 0x7f6afc151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afbfff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6afc154970 0x7f6afc151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54740/0 (socket says 172.21.15.129:54740) 2024-04-25T01:01:59.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afbfff640 1 -- 172.21.15.129:0/1680722370 learned_addr learned my addr 172.21.15.129:0/1680722370 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f180f7fe640 1 -- 172.21.15.129:0/1708523238 learned_addr learned my addr 172.21.15.129:0/1708523238 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:01:59.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f798d810640 1 -- 172.21.15.129:0/1266120845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 msgr2=0x7f79800df000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f798d810640 1 --2- 172.21.15.129:0/1266120845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f79800df000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f798802d700 tx=0x7f79880a5a80 comp rx=0 tx=0).stop 2024-04-25T01:01:59.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f798d810640 1 -- 172.21.15.129:0/1266120845 shutdown_connections 2024-04-25T01:01:59.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f798d810640 1 --2- 172.21.15.129:0/1266120845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f79800df000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f798d810640 1 --2- 172.21.15.129:0/1266120845 >> v2:172.21.15.171:3300/0 conn(0x7f7980047240 0x7f79800479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f798d810640 1 -- 172.21.15.129:0/1266120845 >> 172.21.15.129:0/1266120845 conn(0x7f7980049340 msgr2=0x7f7980049750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afb7fe640 1 --2- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc0a9f90 0x7f6afc06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afbfff640 1 -- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc0a9f90 msgr2=0x7f6afc06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afbfff640 1 --2- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc0a9f90 0x7f6afc06e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.942+0000 7f6afbfff640 1 -- 172.21.15.129:0/1680722370 >> v1:172.21.15.129:6789/0 conn(0x7f6afc0a8a50 legacy=0x7f6afc0c1d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:01:59.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f180e7fc640 1 -- 172.21.15.129:0/1708523238 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929697727 0 0) 0x7f1810152330 con 0x7f181014f840 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f180e7fc640 1 -- 172.21.15.129:0/1708523238 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f8003520 con 0x7f181014f840 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f180ffff640 1 -- 172.21.15.129:0/1708523238 >> v1:172.21.15.171:6789/0 conn(0x7f181014f840 legacy=0x7f18100c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f180e7fc640 1 -- 172.21.15.129:0/1708523238 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684567423 0 0) 0x7f18100c31f0 con 0x7f1810152e10 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f180ffff640 1 -- 172.21.15.129:0/1708523238 >> v1:172.21.15.129:6789/0 conn(0x7f1810152e10 legacy=0x7f18100c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f180ffff640 1 -- 172.21.15.129:0/1708523238 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18101528f0 con 0x7f18100c26e0 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.943+0000 7f6afbfff640 1 -- 172.21.15.129:0/1680722370 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0aaca0 con 0x7f6afc154970 2024-04-25T01:01:59.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.944+0000 7f6b6a87b640 1 Processor -- start 2024-04-25T01:01:59.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.944+0000 7f6b6a87b640 1 -- start start 2024-04-25T01:01:59.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.944+0000 7f798d810640 1 -- 172.21.15.129:0/1266120845 shutdown_connections 2024-04-25T01:01:59.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.944+0000 7f6afbfff640 1 --2- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc154970 0x7f6afc151aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009e50 tx=0x7f6af402a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f9acaceb8b33bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f798d810640 1 -- 172.21.15.129:0/1266120845 wait complete. 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b6a87b640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6b640718c0 0x7f6b64151700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b6a87b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f6b64066ea0 0x7f6b641553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b6a87b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b64151e70 con 0x7f6b64072020 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b6a87b640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f6b640accf0 con 0x7f6b64066ea0 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b6a87b640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f6b64152710 con 0x7f6b640718c0 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b68e18640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f6b64066ea0 0x7f6b641553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b68e18640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f6b64066ea0 0x7f6b641553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38750/0 (socket says 172.21.15.129:38750) 2024-04-25T01:01:59.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6b68e18640 1 -- 172.21.15.129:0/3911412255 learned_addr learned my addr 172.21.15.129:0/3911412255 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.945+0000 7f6afa7fc640 1 -- 172.21.15.129:0/1680722370 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4036070 con 0x7f6afc154970 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f6afa7fc640 1 -- 172.21.15.129:0/1680722370 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af4031600 con 0x7f6afc154970 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f6afa7fc640 1 -- 172.21.15.129:0/1680722370 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40319d0 con 0x7f6afc154970 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f6b68e18640 1 -- 172.21.15.129:0/3911412255 >> v2:172.21.15.171:3300/0 conn(0x7f6b640718c0 msgr2=0x7f6b64151700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f6b68e18640 1 --2- 172.21.15.129:0/3911412255 >> v2:172.21.15.171:3300/0 conn(0x7f6b640718c0 0x7f6b64151700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f6b68e18640 1 -- 172.21.15.129:0/3911412255 >> v1:172.21.15.129:6789/0 conn(0x7f6b64072020 legacy=0x7f6b6414eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:01:59.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f6b68e18640 1 -- 172.21.15.129:0/3911412255 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b64152150 con 0x7f6b64066ea0 2024-04-25T01:01:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.946+0000 7f798d810640 1 Processor -- start 2024-04-25T01:01:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f798d810640 1 -- start start 2024-04-25T01:01:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 -- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc154970 msgr2=0x7f6afc151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 --2- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc154970 0x7f6afc151aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009e50 tx=0x7f6af402a630 comp rx=0 tx=0).stop 2024-04-25T01:01:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 -- 172.21.15.129:0/1680722370 shutdown_connections 2024-04-25T01:01:59.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 --2- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc154970 0x7f6afc151aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 --2- 172.21.15.129:0/1680722370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc0a9f90 0x7f6afc06e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 -- 172.21.15.129:0/1680722370 >> 172.21.15.129:0/1680722370 conn(0x7f6afc06cc80 msgr2=0x7f6afc0a9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 -- 172.21.15.129:0/1680722370 shutdown_connections 2024-04-25T01:01:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b68e18640 1 --2- 172.21.15.129:0/3911412255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64066ea0 0x7f6b641553b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ac640 tx=0x7f6b5c02a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccda5a1a733cfdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.947+0000 7f6b00dea640 1 -- 172.21.15.129:0/1680722370 wait complete. 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.948+0000 7f6b62ffd640 1 -- 172.21.15.129:0/3911412255 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c037070 con 0x7f6b64066ea0 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.948+0000 7f6b62ffd640 1 -- 172.21.15.129:0/3911412255 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5c0323f0 con 0x7f6b64066ea0 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 -- 172.21.15.129:0/3911412255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64066ea0 msgr2=0x7f6b641553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/3911412255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64066ea0 0x7f6b641553b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ac640 tx=0x7f6b5c02a840 comp rx=0 tx=0).stop 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 -- 172.21.15.129:0/3911412255 shutdown_connections 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/3911412255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64066ea0 0x7f6b641553b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/3911412255 >> v2:172.21.15.171:3300/0 conn(0x7f6b640718c0 0x7f6b64151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 -- 172.21.15.129:0/3911412255 >> 172.21.15.129:0/3911412255 conn(0x7f6b6406b070 msgr2=0x7f6b6406b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.948+0000 7f798d810640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f798003fb00 0x7f798004afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.948+0000 7f798d810640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 0x7f79801775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f79867fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 0x7f79801775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f79867fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 0x7f79801775d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38754/0 (socket says 172.21.15.129:38754) 2024-04-25T01:01:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f79867fc640 1 -- 172.21.15.129:0/1005064242 learned_addr learned my addr 172.21.15.129:0/1005064242 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f798d810640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f7980177b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b00dea640 1 Processor -- start 2024-04-25T01:01:59.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f798d810640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f798004a200 con 0x7f7980047240 2024-04-25T01:01:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f798d810640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f798004a080 con 0x7f798003fb00 2024-04-25T01:01:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f798d810640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f79800dc400 con 0x7f79800deb90 2024-04-25T01:01:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f79877fe640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f7980177b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b00dea640 1 -- start start 2024-04-25T01:01:59.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f7986ffd640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f798003fb00 0x7f798004afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f180ffff640 1 --2- 172.21.15.129:0/1708523238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18100c26e0 0x7f181015ebd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18000097d0 tx=0x7f180002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e260bc12f2f634 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f79877fe640 1 -- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f798003fb00 msgr2=0x7f798004afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f79877fe640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f798003fb00 0x7f798004afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f79877fe640 1 -- 172.21.15.129:0/1005064242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 msgr2=0x7f79801775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f79877fe640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 0x7f79801775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.950+0000 7f79877fe640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7980042e10 con 0x7f79800deb90 2024-04-25T01:01:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b00dea640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 0x7f6afc1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b00dea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6afc0a9f90 0x7f6afc1415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b00dea640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc154970 0x7f6afc0c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 0x7f6afc1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 0x7f6afc1410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54768/0 (socket says 172.21.15.129:54768) 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 -- 172.21.15.129:0/3468213135 learned_addr learned my addr 172.21.15.129:0/3468213135 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc151fe0 con 0x7f6afc0a9f90 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc0aaf80 con 0x7f6afc0a8a50 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f6afc152160 con 0x7f6afc154970 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f79867fc640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 0x7f79801775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:01:59.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afbfff640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc154970 0x7f6afc0c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f79877fe640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f7980177b10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79880698b0 tx=0x7f7988068f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 -- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc154970 msgr2=0x7f6afc0c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc154970 0x7f6afc0c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 -- 172.21.15.129:0/3468213135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6afc0a9f90 msgr2=0x7f6afc1415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6afc0a9f90 0x7f6afc1415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6afb7fe640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0a5f90 con 0x7f6afc0a8a50 2024-04-25T01:01:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 -- 172.21.15.129:0/3911412255 shutdown_connections 2024-04-25T01:01:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.949+0000 7f6b6a87b640 1 -- 172.21.15.129:0/3911412255 wait complete. 2024-04-25T01:01:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b6a87b640 1 Processor -- start 2024-04-25T01:01:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.951+0000 7f6b6a87b640 1 -- start start 2024-04-25T01:01:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f6afb7fe640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 0x7f6afc1410b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9e0 tx=0x7f6aec00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f180e7fc640 1 -- 172.21.15.129:0/1708523238 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800037070 con 0x7f18100c26e0 2024-04-25T01:01:59.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f180e7fc640 1 -- 172.21.15.129:0/1708523238 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1800032440 con 0x7f18100c26e0 2024-04-25T01:01:59.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f180e7fc640 1 -- 172.21.15.129:0/1708523238 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800032810 con 0x7f18100c26e0 2024-04-25T01:01:59.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec017070 con 0x7f6afc0a8a50 2024-04-25T01:01:59.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec00c9d0 con 0x7f6afc0a8a50 2024-04-25T01:01:59.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00cdc0 con 0x7f6afc0a8a50 2024-04-25T01:01:59.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f1815ce1640 1 -- 172.21.15.129:0/1708523238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18100c26e0 msgr2=0x7f181015ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880b1e40 con 0x7f79800deb90 2024-04-25T01:01:59.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6afc1592b0 con 0x7f6afc0a8a50 2024-04-25T01:01:59.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f1815ce1640 1 --2- 172.21.15.129:0/1708523238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18100c26e0 0x7f181015ebd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18000097d0 tx=0x7f180002a180 comp rx=0 tx=0).stop 2024-04-25T01:01:59.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f6b6a87b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b640718c0 0x7f6b641505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.952+0000 7f6b6a87b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 0x7f6b6412be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 0x7f6b6412be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 0x7f6b6412be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38770/0 (socket says 172.21.15.129:38770) 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b640718c0 0x7f6b641505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 -- 172.21.15.129:0/1858041343 learned_addr learned my addr 172.21.15.129:0/1858041343 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b6408b3c0 0x7f6b6412c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64154f20 con 0x7f6b64072020 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640672f0 con 0x7f6b640718c0 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f6b64067470 con 0x7f6b6408b3c0 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 -- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b6408b3c0 msgr2=0x7f6b6412c3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b6408b3c0 0x7f6b6412c3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 -- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b640718c0 msgr2=0x7f6b641505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b640718c0 0x7f6b641505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b637fe640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6413b680 con 0x7f6b64072020 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b637fe640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 0x7f6b6412be70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400b6d0 tx=0x7f6b5400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b63fff640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b640718c0 0x7f6b641505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54019070 con 0x7f6b64072020 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5400c7e0 con 0x7f6b64072020 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400cbd0 con 0x7f6b64072020 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b641517a0 con 0x7f6b64072020 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6413f820 con 0x7f6b64072020 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f1815ce1640 1 -- 172.21.15.129:0/1708523238 shutdown_connections 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f1815ce1640 1 --2- 172.21.15.129:0/1708523238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18100c26e0 0x7f181015ebd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f1815ce1640 1 -- 172.21.15.129:0/1708523238 >> 172.21.15.129:0/1708523238 conn(0x7f18100c7920 msgr2=0x7f18100c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f1815ce1640 1 -- 172.21.15.129:0/1708523238 shutdown_connections 2024-04-25T01:01:59.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f798d810640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79800430b0 con 0x7f79800deb90 2024-04-25T01:01:59.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.954+0000 7f798d810640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801743f0 con 0x7f79800deb90 2024-04-25T01:01:59.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.953+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6afc0c43d0 con 0x7f6afc0a8a50 2024-04-25T01:01:59.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6afc075440 con 0x7f6afc0a8a50 2024-04-25T01:01:59.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79880c1070 con 0x7f79800deb90 2024-04-25T01:01:59.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880bc610 con 0x7f79800deb90 2024-04-25T01:01:59.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6aec01a070 con 0x7f6afc0a8a50 2024-04-25T01:01:59.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79880b9030 con 0x7f79800deb90 2024-04-25T01:01:59.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b5401c070 con 0x7f6b64072020 2024-04-25T01:01:59.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.956+0000 7f1815ce1640 1 -- 172.21.15.129:0/1708523238 wait complete. 2024-04-25T01:01:59.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f6b60ff9640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6b5006bf00 0x7f6b5006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f1815ce1640 1 Processor -- start 2024-04-25T01:01:59.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f796ffff640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f797406bf30 0x7f797406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7988127e00 con 0x7f79800deb90 2024-04-25T01:01:59.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f7986ffd640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f797406bf30 0x7f797406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7968005290 con 0x7f79800deb90 2024-04-25T01:01:59.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6b5408a6a0 con 0x7f6b64072020 2024-04-25T01:01:59.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f6b63fff640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6b5006bf00 0x7f6b5006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b44005290 con 0x7f6b64072020 2024-04-25T01:01:59.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f6b63fff640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6b5006bf00 0x7f6b5006e3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ad4d0 tx=0x7f6b58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f7986ffd640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f797406bf30 0x7f797406e3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f6ae3fff640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6ae806bf00 0x7f6ae806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6aec08b0c0 con 0x7f6afc0a8a50 2024-04-25T01:01:59.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.957+0000 7f6afaffd640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6ae806bf00 0x7f6ae806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f6afaffd640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6ae806bf00 0x7f6ae806e3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6afc1631d0 tx=0x7f6af00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.958+0000 7f1815ce1640 1 -- start start 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f1815ce1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18100c26e0 0x7f181014e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f1815ce1640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f181014f840 0x7f181013ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f1815ce1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 0x7f181013bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f1815ce1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1810072780 con 0x7f1810152e10 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f1815ce1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1810072600 con 0x7f18100c26e0 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f1815ce1640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f18100c5f30 con 0x7f181014f840 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f180ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 0x7f181013bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f180ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 0x7f181013bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38772/0 (socket says 172.21.15.129:38772) 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.959+0000 7f180ffff640 1 -- 172.21.15.129:0/2085391562 learned_addr learned my addr 172.21.15.129:0/2085391562 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180effd640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f181014f840 0x7f181013ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180f7fe640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18100c26e0 0x7f181014e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180ffff640 1 -- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f181014f840 msgr2=0x7f181013ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180ffff640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f181014f840 0x7f181013ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180ffff640 1 -- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18100c26e0 msgr2=0x7f181014e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180ffff640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18100c26e0 0x7f181014e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180ffff640 1 -- 172.21.15.129:0/2085391562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181013c4f0 con 0x7f1810152e10 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.960+0000 7f180ffff640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 0x7f181013bfb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18000098f0 tx=0x7f18000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.965+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6aec059150 con 0x7f6afc0a8a50 2024-04-25T01:01:59.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.967+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f79880b7040 con 0x7f79800deb90 2024-04-25T01:01:59.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b54058d80 con 0x7f6b64072020 2024-04-25T01:01:59.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800037050 con 0x7f1810152e10 2024-04-25T01:01:59.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181014f020 con 0x7f1810152e10 2024-04-25T01:01:59.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181014d720 con 0x7f1810152e10 2024-04-25T01:01:59.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1800043070 con 0x7f1810152e10 2024-04-25T01:01:59.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800037050 con 0x7f1810152e10 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.967+0000 7fa86e4d6640 1 Processor -- start 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.967+0000 7fa86e4d6640 1 -- start start 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7fa86e4d6640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 0x7fa8680728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7fa86e4d6640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa868159780 0x7fa8680c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7fa86e4d6640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa868159ef0 con 0x7fa86806b2e0 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7fa86e4d6640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fa86806bf70 con 0x7fa868159780 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.968+0000 7fa86e4d6640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa86815a790 con 0x7fa8681571b0 2024-04-25T01:01:59.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 0x7fa8680728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 0x7fa8680728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47868/0 (socket says 172.21.15.129:47868) 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 -- 172.21.15.129:0/1030313127 learned_addr learned my addr 172.21.15.129:0/1030313127 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 -- 172.21.15.129:0/1030313127 >> v2:172.21.15.171:3300/0 conn(0x7fa868159780 msgr2=0x7fa8680c5e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 --2- 172.21.15.129:0/1030313127 >> v2:172.21.15.171:3300/0 conn(0x7fa868159780 0x7fa8680c5e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 -- 172.21.15.129:0/1030313127 >> v1:172.21.15.129:6789/0 conn(0x7fa86806b2e0 legacy=0x7fa868156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.969+0000 7fa8677fe640 1 -- 172.21.15.129:0/1030313127 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86815a1d0 con 0x7fa8681571b0 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.970+0000 7fa8677fe640 1 --2- 172.21.15.129:0/1030313127 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 0x7fa8680728b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa858009870 tx=0x7fa85802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b79ccfc351a7e28b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.971+0000 7fa866ffd640 1 -- 172.21.15.129:0/1030313127 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858037070 con 0x7fa8681571b0 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.971+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18000308b0 con 0x7f1810152e10 2024-04-25T01:01:59.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.971+0000 7f17f7fff640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f17fc06c230 0x7f17fc06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.971+0000 7fa866ffd640 1 -- 172.21.15.129:0/1030313127 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858032440 con 0x7fa8681571b0 2024-04-25T01:01:59.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f18000306b0 con 0x7f1810152e10 2024-04-25T01:01:59.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.973+0000 7f180f7fe640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f17fc06c230 0x7f17fc06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.974+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f0005290 con 0x7f1810152e10 2024-04-25T01:01:59.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7fa86e4d6640 1 -- 172.21.15.129:0/1030313127 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 msgr2=0x7fa8680728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/1030313127 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 0x7fa8680728b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa858009870 tx=0x7fa85802a210 comp rx=0 tx=0).stop 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7fa86e4d6640 1 -- 172.21.15.129:0/1030313127 shutdown_connections 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/1030313127 >> v2:172.21.15.171:3300/0 conn(0x7fa868159780 0x7fa8680c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/1030313127 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8681571b0 0x7fa8680728b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.972+0000 7fa86e4d6640 1 -- 172.21.15.129:0/1030313127 >> 172.21.15.129:0/1030313127 conn(0x7fa8680c4410 msgr2=0x7fa8680c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.973+0000 7fa86e4d6640 1 -- 172.21.15.129:0/1030313127 shutdown_connections 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.973+0000 7fa86e4d6640 1 -- 172.21.15.129:0/1030313127 wait complete. 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.974+0000 7fa86e4d6640 1 Processor -- start 2024-04-25T01:01:59.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.975+0000 7fa86e4d6640 1 -- start start 2024-04-25T01:01:59.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.985+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1800079e30 con 0x7f1810152e10 2024-04-25T01:01:59.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.989+0000 7f180f7fe640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f17fc06c230 0x7f17fc06e6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18080012e0 tx=0x7f1808001740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.993+0000 7fa86e4d6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa86806b2e0 0x7fa86806e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.993+0000 7fa86e4d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 0x7fa8681746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.993+0000 7fa86e4d6640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa868174c20 0x7fa8681462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.993+0000 7fa86e4d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa868159c00 con 0x7fa868159780 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.993+0000 7fa86e4d6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680c59f0 con 0x7fa86806b2e0 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.993+0000 7fa86e4d6640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa86806b6c0 con 0x7fa868174c20 2024-04-25T01:01:59.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.994+0000 7fa86ca73640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa868174c20 0x7fa8681462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.996+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 0x7fa8681746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:01:59.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.996+0000 7fa8677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 0x7fa8681746e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38780/0 (socket says 172.21.15.129:38780) 2024-04-25T01:01:59.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.996+0000 7fa8677fe640 1 -- 172.21.15.129:0/3461556377 learned_addr learned my addr 172.21.15.129:0/3461556377 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:01:59.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.997+0000 7fa8677fe640 1 -- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa868174c20 msgr2=0x7fa8681462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:01:59.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.997+0000 7fa8677fe640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa868174c20 0x7fa8681462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.997+0000 7fa8677fe640 1 -- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa86806b2e0 msgr2=0x7fa86806e540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:01:59.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.997+0000 7fa8677fe640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa86806b2e0 0x7fa86806e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:01:59.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.997+0000 7fa8677fe640 1 -- 172.21.15.129:0/3461556377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8680a8cf0 con 0x7fa868159780 2024-04-25T01:01:59.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.997+0000 7fa8677fe640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 0x7fa8681746e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa858009990 tx=0x7fa8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:01:59.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.998+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858041020 con 0x7fa868159780 2024-04-25T01:01:59.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.998+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa868145ad0 con 0x7fa868159780 2024-04-25T01:01:59.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:01:59.998+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8680a8870 con 0x7fa868159780 2024-04-25T01:02:00.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.001+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858032440 con 0x7fa868159780 2024-04-25T01:02:00.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.001+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858037040 con 0x7fa868159780 2024-04-25T01:02:00.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.001+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa858044020 con 0x7fa868159780 2024-04-25T01:02:00.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.001+0000 7fa864ff9640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa85406bf30 0x7fa85406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.003+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868175030 con 0x7fa868159780 2024-04-25T01:02:00.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.003+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa8580abb50 con 0x7fa868159780 2024-04-25T01:02:00.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.012+0000 7fa867fff640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa85406bf30 0x7fa85406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.012+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa868175030 con 0x7fa868159780 2024-04-25T01:02:00.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.045+0000 7fa867fff640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa85406bf30 0x7fa85406e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0014e0 tx=0x7fa85c0079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.293+0000 7f0625bfb640 1 -- 172.21.15.129:0/2401532095 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f060c005850 con 0x7f062006da00 2024-04-25T01:02:00.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.293+0000 7f0607fff640 1 -- 172.21.15.129:0/2401532095 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0610058aa0 con 0x7f062006da00 2024-04-25T01:02:00.294 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:02:00.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.297+0000 7f0605ffb640 1 -- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f060806c070 msgr2=0x7f060806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.297+0000 7f0605ffb640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f060806c070 0x7f060806e530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0620160650 tx=0x7f061800a040 comp rx=0 tx=0).stop 2024-04-25T01:02:00.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.297+0000 7f0605ffb640 1 -- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 msgr2=0x7f06200c0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.297+0000 7f0605ffb640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 0x7f06200c0820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f061000c600 tx=0x7f061000cac0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.297+0000 7f0605ffb640 1 -- 172.21.15.129:0/2401532095 shutdown_connections 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.297+0000 7f0605ffb640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f060806c070 0x7f060806e530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.298+0000 7f0605ffb640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0620165880 0x7f06200bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.298+0000 7f0605ffb640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06200c2d90 0x7f06200bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.298+0000 7f0605ffb640 1 --2- 172.21.15.129:0/2401532095 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062006da00 0x7f06200c0820 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.298+0000 7f0605ffb640 1 -- 172.21.15.129:0/2401532095 >> 172.21.15.129:0/2401532095 conn(0x7f06200c67b0 msgr2=0x7f06200a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.298+0000 7f0605ffb640 1 -- 172.21.15.129:0/2401532095 shutdown_connections 2024-04-25T01:02:00.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.298+0000 7f0605ffb640 1 -- 172.21.15.129:0/2401532095 wait complete. 2024-04-25T01:02:00.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T01:02:00.316 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.326+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0190005580 con 0x7f01a80bbf70 2024-04-25T01:02:00.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.326+0000 7f019e7fc640 1 -- 172.21.15.129:0/2595741528 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01a0079a20 con 0x7f01a80bbf70 2024-04-25T01:02:00.327 INFO:teuthology.orchestra.run.smithi129.stdout:55834574859 2024-04-25T01:02:00.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.329+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f018c06bf00 msgr2=0x7f018c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.329+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f018c06bf00 0x7f018c06e3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f01a815e200 tx=0x7f01a4009060 comp rx=0 tx=0).stop 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.329+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 msgr2=0x7f01a81f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.329+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 0x7f01a81f78c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b290 tx=0x7f01a0004a40 comp rx=0 tx=0).stop 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.329+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 shutdown_connections 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f018c06bf00 0x7f018c06e3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f01a8152070 0x7f01a8154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01a80bbf70 0x7f01a81f78c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 --2- 172.21.15.129:0/2595741528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a80ac090 0x7f01a8073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 >> 172.21.15.129:0/2595741528 conn(0x7f01a80a86b0 msgr2=0x7f01a80ab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 shutdown_connections 2024-04-25T01:02:00.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.330+0000 7f01b01e7640 1 -- 172.21.15.129:0/2595741528 wait complete. 2024-04-25T01:02:00.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.339+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3574005580 con 0x7f35901eeb60 2024-04-25T01:02:00.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.340+0000 7f3587fff640 1 -- 172.21.15.129:0/3368843012 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3580059290 con 0x7f35901eeb60 2024-04-25T01:02:00.340 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f357006be30 msgr2=0x7f357006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f357006be30 0x7f357006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3588006fd0 tx=0x7f3588008040 comp rx=0 tx=0).stop 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 msgr2=0x7f359008a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 0x7f359008a680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f358000caa0 tx=0x7f358000cf60 comp rx=0 tx=0).stop 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 shutdown_connections 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f357006be30 0x7f357006e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f35900ac600 0x7f3590072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35901eeb60 0x7f359008a680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 --2- 172.21.15.129:0/3368843012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359006dc00 0x7f35901ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 >> 172.21.15.129:0/3368843012 conn(0x7f35900a86b0 msgr2=0x7f35900ab920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 shutdown_connections 2024-04-25T01:02:00.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.342+0000 7f3595eb3640 1 -- 172.21.15.129:0/3368843012 wait complete. 2024-04-25T01:02:00.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T01:02:00.347 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.347+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7968005580 con 0x7f79800deb90 2024-04-25T01:02:00.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.348+0000 7f796ffff640 1 -- 172.21.15.129:0/1005064242 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f79880ad030 con 0x7f79800deb90 2024-04-25T01:02:00.348 INFO:teuthology.orchestra.run.smithi129.stdout:55834574857 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f797406bf30 msgr2=0x7f797406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f797406bf30 0x7f797406e3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f797c006fd0 tx=0x7f797c008040 comp rx=0 tx=0).stop 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 msgr2=0x7f7980177b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f7980177b10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79880698b0 tx=0x7f7988068f20 comp rx=0 tx=0).stop 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 shutdown_connections 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f797406bf30 0x7f797406e3f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f79800deb90 0x7f7980177b10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7980047240 0x7f79801775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 --2- 172.21.15.129:0/1005064242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f798003fb00 0x7f798004afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 >> 172.21.15.129:0/1005064242 conn(0x7f7980049340 msgr2=0x7f798002d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 shutdown_connections 2024-04-25T01:02:00.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.350+0000 7f796dffb640 1 -- 172.21.15.129:0/1005064242 wait complete. 2024-04-25T01:02:00.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-25T01:02:00.356 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2024-04-25T01:02:00.362 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.383+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6afc06de70 con 0x7f6afc0a8a50 2024-04-25T01:02:00.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.384+0000 7f6ae3fff640 1 -- 172.21.15.129:0/3468213135 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6aec05d450 con 0x7f6afc0a8a50 2024-04-25T01:02:00.385 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:02:00.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6ae806bf00 msgr2=0x7f6ae806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6ae806bf00 0x7f6ae806e3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6afc1631d0 tx=0x7f6af00079d0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 msgr2=0x7f6afc1410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 0x7f6afc1410b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9e0 tx=0x7f6aec00eea0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 shutdown_connections 2024-04-25T01:02:00.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6ae806bf00 0x7f6ae806e3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6afc154970 0x7f6afc0c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6afc0a9f90 0x7f6afc1415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 --2- 172.21.15.129:0/3468213135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6afc0a8a50 0x7f6afc1410b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 >> 172.21.15.129:0/3468213135 conn(0x7f6afc06cc80 msgr2=0x7f6afc0a6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 shutdown_connections 2024-04-25T01:02:00.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.387+0000 7f6b00dea640 1 -- 172.21.15.129:0/3468213135 wait complete. 2024-04-25T01:02:00.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.391+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6b44005580 con 0x7f6b64072020 2024-04-25T01:02:00.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.391+0000 7f6b60ff9640 1 -- 172.21.15.129:0/1858041343 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b54058680 con 0x7f6b64072020 2024-04-25T01:02:00.392 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:02:00.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6b5006bf00 msgr2=0x7f6b5006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6b5006bf00 0x7f6b5006e3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ad4d0 tx=0x7f6b58008040 comp rx=0 tx=0).stop 2024-04-25T01:02:00.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 msgr2=0x7f6b6412be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 0x7f6b6412be70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400b6d0 tx=0x7f6b5400bb90 comp rx=0 tx=0).stop 2024-04-25T01:02:00.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 shutdown_connections 2024-04-25T01:02:00.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f6b5006bf00 0x7f6b5006e3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b6408b3c0 0x7f6b6412c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6b64072020 0x7f6b6412be70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 --2- 172.21.15.129:0/1858041343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b640718c0 0x7f6b641505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 >> 172.21.15.129:0/1858041343 conn(0x7f6b6406b070 msgr2=0x7f6b6406ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 shutdown_connections 2024-04-25T01:02:00.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.393+0000 7f6b6a87b640 1 -- 172.21.15.129:0/1858041343 wait complete. 2024-04-25T01:02:00.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.398+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f17f0005850 con 0x7f1810152e10 2024-04-25T01:02:00.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.398+0000 7f17f7fff640 1 -- 172.21.15.129:0/2085391562 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1800079e30 con 0x7f1810152e10 2024-04-25T01:02:00.399 INFO:teuthology.orchestra.run.smithi129.stdout:55834574859 2024-04-25T01:02:00.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f17fc06c230 msgr2=0x7f17fc06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f17fc06c230 0x7f17fc06e6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18080012e0 tx=0x7f1808001740 comp rx=0 tx=0).stop 2024-04-25T01:02:00.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 msgr2=0x7f181013bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 0x7f181013bfb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18000098f0 tx=0x7f18000046c0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 shutdown_connections 2024-04-25T01:02:00.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f17fc06c230 0x7f17fc06e6f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f181014f840 0x7f181013ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1810152e10 0x7f181013bfb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 --2- 172.21.15.129:0/2085391562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18100c26e0 0x7f181014e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 >> 172.21.15.129:0/2085391562 conn(0x7f18100c7920 msgr2=0x7f1810074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 shutdown_connections 2024-04-25T01:02:00.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.401+0000 7f1815ce1640 1 -- 172.21.15.129:0/2085391562 wait complete. 2024-04-25T01:02:00.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-04-25T01:02:00.404 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-25T01:02:00.406 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.409+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa868075550 con 0x7fa868159780 2024-04-25T01:02:00.409 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.409+0000 7fa864ff9640 1 -- 172.21.15.129:0/3461556377 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa85804b030 con 0x7fa868159780 2024-04-25T01:02:00.410 INFO:teuthology.orchestra.run.smithi129.stdout:55834574858 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa85406bf30 msgr2=0x7fa85406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa85406bf30 0x7fa85406e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0014e0 tx=0x7fa85c0079d0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 msgr2=0x7fa8681746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 0x7fa8681746e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa858009990 tx=0x7fa8580046c0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 shutdown_connections 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa85406bf30 0x7fa85406e3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa868174c20 0x7fa8681462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa868159780 0x7fa8681746e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 --2- 172.21.15.129:0/3461556377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa86806b2e0 0x7fa86806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 >> 172.21.15.129:0/3461556377 conn(0x7fa8680c4410 msgr2=0x7fa8680c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 shutdown_connections 2024-04-25T01:02:00.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.411+0000 7fa86e4d6640 1 -- 172.21.15.129:0/3461556377 wait complete. 2024-04-25T01:02:00.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-25T01:02:00.414 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-04-25T01:02:00.423 DEBUG:teuthology.parallel:result is None 2024-04-25T01:02:00.423 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T01:02:00.423 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:02:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.560+0000 7f8301cfc640 1 Processor -- start 2024-04-25T01:02:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.560+0000 7f8301cfc640 1 -- start start 2024-04-25T01:02:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f8301cfc640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f8301cfc640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f82fc06eba0 0x7f82fc072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f8301cfc640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82fc150c30 con 0x7f82fc0671c0 2024-04-25T01:02:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f8301cfc640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f82fc06f000 con 0x7f82fc06eba0 2024-04-25T01:02:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f8301cfc640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc0c1bf0 con 0x7f82fc150850 2024-04-25T01:02:00.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fb7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fbfff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f82fc06eba0 0x7f82fc072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fb7fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:47890/0 (socket says 172.21.15.129:47890) 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82faffd640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f82fc0671c0 0x7f82fc0c2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54072/0 (socket says 172.21.15.129:54072) 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fbfff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f82fc06eba0 0x7f82fc072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54826/0 (socket says 172.21.15.129:54826) 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fb7fe640 1 -- 172.21.15.129:0/3960322117 learned_addr learned my addr 172.21.15.129:0/3960322117 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fbfff640 1 -- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 msgr2=0x7f82fc071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fbfff640 1 --2- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fbfff640 1 -- 172.21.15.129:0/3960322117 >> v1:172.21.15.171:6789/0 conn(0x7f82fc0671c0 legacy=0x7f82fc0c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fbfff640 1 -- 172.21.15.129:0/3960322117 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc0c2a70 con 0x7f82fc06eba0 2024-04-25T01:02:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.561+0000 7f82fa7fc640 1 -- 172.21.15.129:0/3960322117 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531248650 0 0) 0x7f82fc150c30 con 0x7f82fc0671c0 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.562+0000 7f82fbfff640 1 --2- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc06eba0 0x7f82fc072e80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009d90 tx=0x7f82f402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9f80f9191e79e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.562+0000 7f82fa7fc640 1 -- 172.21.15.129:0/3960322117 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4007740 con 0x7f82fc06eba0 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.562+0000 7f82fa7fc640 1 -- 172.21.15.129:0/3960322117 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f40078e0 con 0x7f82fc06eba0 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.562+0000 7f82fa7fc640 1 -- 172.21.15.129:0/3960322117 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4007cb0 con 0x7f82fc06eba0 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 -- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc06eba0 msgr2=0x7f82fc072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 --2- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc06eba0 0x7f82fc072e80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009d90 tx=0x7f82f402a660 comp rx=0 tx=0).stop 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 -- 172.21.15.129:0/3960322117 shutdown_connections 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 --2- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc06eba0 0x7f82fc072e80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 --2- 172.21.15.129:0/3960322117 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 -- 172.21.15.129:0/3960322117 >> 172.21.15.129:0/3960322117 conn(0x7f82fc0c71f0 msgr2=0x7f82fc0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 -- 172.21.15.129:0/3960322117 shutdown_connections 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.563+0000 7f8301cfc640 1 -- 172.21.15.129:0/3960322117 wait complete. 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.564+0000 7f8301cfc640 1 Processor -- start 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.564+0000 7f8301cfc640 1 -- start start 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc0671c0 0x7f82fc145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 0x7f82fc1df500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0c3310 con 0x7f82fc06eba0 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0c3030 con 0x7f82fc0671c0 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc06c8e0 con 0x7f82fc150850 2024-04-25T01:02:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 0x7f82fc1df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82fb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc0671c0 0x7f82fc145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 0x7f82fc1df500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38792/0 (socket says 172.21.15.129:38792) 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 -- 172.21.15.129:0/2736464780 learned_addr learned my addr 172.21.15.129:0/2736464780 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82fbfff640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 -- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 msgr2=0x7f82fc14bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc14bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 -- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc0671c0 msgr2=0x7f82fc145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc0671c0 0x7f82fc145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 -- 172.21.15.129:0/2736464780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc14ccd0 con 0x7f82fc06eba0 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82fb7fe640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc0671c0 0x7f82fc145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82faffd640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 0x7f82fc1df500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c4d0 tx=0x7f82ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec018070 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec007500 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.565+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc1ddd60 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.566+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004a20 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.566+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1ed320 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.566+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc075550 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.567+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82ec01b070 con 0x7f82fc06eba0 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.567+0000 7f82e3fff640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f82e806be30 0x7f82e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.567+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f82ec08a470 con 0x7f82fc06eba0 2024-04-25T01:02:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.568+0000 7f82fb7fe640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f82e806be30 0x7f82e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.568+0000 7f82fb7fe640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f82e806be30 0x7f82e806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009f20 tx=0x7f82f0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.571+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82ec058490 con 0x7f82fc06eba0 2024-04-25T01:02:00.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.740+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82fc14d3f0 con 0x7f82e806be30 2024-04-25T01:02:00.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.748+0000 7f82e3fff640 1 -- 172.21.15.129:0/2736464780 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f82fc14d3f0 con 0x7f82e806be30 2024-04-25T01:02:00.753 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:00.755 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:02:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f82e806be30 msgr2=0x7f82e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f82e806be30 0x7f82e806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009f20 tx=0x7f82f0009240 comp rx=0 tx=0).stop 2024-04-25T01:02:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 msgr2=0x7f82fc1df500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 0x7f82fc1df500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c4d0 tx=0x7f82ec00c990 comp rx=0 tx=0).stop 2024-04-25T01:02:00.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 shutdown_connections 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f82e806be30 0x7f82e806e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f82fc150850 0x7f82fc14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82fc06eba0 0x7f82fc1df500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 --2- 172.21.15.129:0/2736464780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82fc0671c0 0x7f82fc145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 >> 172.21.15.129:0/2736464780 conn(0x7f82fc0c71f0 msgr2=0x7f82fc0c1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 shutdown_connections 2024-04-25T01:02:00.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.752+0000 7f8301cfc640 1 -- 172.21.15.129:0/2736464780 wait complete. 2024-04-25T01:02:00.763 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T01:02:00.672306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219492,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755228,"statfs":{"total":773094113280,"available":772869353472,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003706"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468886+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:01:39.468886+0000","last_peered":"2024-04-25T01:01:39.468886+0000","last_clean":"2024-04-25T01:01:39.468886+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:01:39.468886+0000","last_undegraded":"2024-04-25T01:01:39.468886+0000","last_fullsized":"2024-04-25T01:01:39.468886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415373+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:01:38.415373+0000","last_peered":"2024-04-25T01:01:38.415373+0000","last_clean":"2024-04-25T01:01:38.415373+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:01:38.415373+0000","last_undegraded":"2024-04-25T01:01:38.415373+0000","last_fullsized":"2024-04-25T01:01:38.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414835+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:01:38.414835+0000","last_peered":"2024-04-25T01:01:38.414835+0000","last_clean":"2024-04-25T01:01:38.414835+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:01:38.414835+0000","last_undegraded":"2024-04-25T01:01:38.414835+0000","last_fullsized":"2024-04-25T01:01:38.414835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468718+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:01:39.468718+0000","last_peered":"2024-04-25T01:01:39.468718+0000","last_clean":"2024-04-25T01:01:39.468718+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:01:39.468718+0000","last_undegraded":"2024-04-25T01:01:39.468718+0000","last_fullsized":"2024-04-25T01:01:39.468718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415475+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:01:38.415475+0000","last_peered":"2024-04-25T01:01:38.415475+0000","last_clean":"2024-04-25T01:01:38.415475+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:01:38.415475+0000","last_undegraded":"2024-04-25T01:01:38.415475+0000","last_fullsized":"2024-04-25T01:01:38.415475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417116+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:01:38.417116+0000","last_peered":"2024-04-25T01:01:38.417116+0000","last_clean":"2024-04-25T01:01:38.417116+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:01:38.417116+0000","last_undegraded":"2024-04-25T01:01:38.417116+0000","last_fullsized":"2024-04-25T01:01:38.417116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459319+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:01:38.459319+0000","last_peered":"2024-04-25T01:01:38.459319+0000","last_clean":"2024-04-25T01:01:38.459319+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:01:38.459319+0000","last_undegraded":"2024-04-25T01:01:38.459319+0000","last_fullsized":"2024-04-25T01:01:38.459319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416961+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:01:38.416961+0000","last_peered":"2024-04-25T01:01:38.416961+0000","last_clean":"2024-04-25T01:01:38.416961+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:01:38.416961+0000","last_undegraded":"2024-04-25T01:01:38.416961+0000","last_fullsized":"2024-04-25T01:01:38.416961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416812+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:01:38.416812+0000","last_peered":"2024-04-25T01:01:38.416812+0000","last_clean":"2024-04-25T01:01:38.416812+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:01:38.416812+0000","last_undegraded":"2024-04-25T01:01:38.416812+0000","last_fullsized":"2024-04-25T01:01:38.416812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459435+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:01:38.459435+0000","last_peered":"2024-04-25T01:01:38.459435+0000","last_clean":"2024-04-25T01:01:38.459435+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:01:38.459435+0000","last_undegraded":"2024-04-25T01:01:38.459435+0000","last_fullsized":"2024-04-25T01:01:38.459435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416306+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:01:38.416306+0000","last_peered":"2024-04-25T01:01:38.416306+0000","last_clean":"2024-04-25T01:01:38.416306+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:01:38.416306+0000","last_undegraded":"2024-04-25T01:01:38.416306+0000","last_fullsized":"2024-04-25T01:01:38.416306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097726+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:01:39.097726+0000","last_peered":"2024-04-25T01:01:39.097726+0000","last_clean":"2024-04-25T01:01:39.097726+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:01:39.097726+0000","last_undegraded":"2024-04-25T01:01:39.097726+0000","last_fullsized":"2024-04-25T01:01:39.097726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416250+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:01:38.416250+0000","last_peered":"2024-04-25T01:01:38.416250+0000","last_clean":"2024-04-25T01:01:38.416250+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:01:38.416250+0000","last_undegraded":"2024-04-25T01:01:38.416250+0000","last_fullsized":"2024-04-25T01:01:38.416250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416610+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:01:38.416610+0000","last_peered":"2024-04-25T01:01:38.416610+0000","last_clean":"2024-04-25T01:01:38.416610+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:01:38.416610+0000","last_undegraded":"2024-04-25T01:01:38.416610+0000","last_fullsized":"2024-04-25T01:01:38.416610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415556+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:01:38.415556+0000","last_peered":"2024-04-25T01:01:38.415556+0000","last_clean":"2024-04-25T01:01:38.415556+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:01:38.415556+0000","last_undegraded":"2024-04-25T01:01:38.415556+0000","last_fullsized":"2024-04-25T01:01:38.415556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551986+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:01:38.551986+0000","last_peered":"2024-04-25T01:01:38.551986+0000","last_clean":"2024-04-25T01:01:38.551986+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:01:38.551986+0000","last_undegraded":"2024-04-25T01:01:38.551986+0000","last_fullsized":"2024-04-25T01:01:38.551986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415994+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:01:38.415994+0000","last_peered":"2024-04-25T01:01:38.415994+0000","last_clean":"2024-04-25T01:01:38.415994+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:01:38.415994+0000","last_undegraded":"2024-04-25T01:01:38.415994+0000","last_fullsized":"2024-04-25T01:01:38.415994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415087+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:01:38.415087+0000","last_peered":"2024-04-25T01:01:38.415087+0000","last_clean":"2024-04-25T01:01:38.415087+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:01:38.415087+0000","last_undegraded":"2024-04-25T01:01:38.415087+0000","last_fullsized":"2024-04-25T01:01:38.415087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551639+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:01:38.551639+0000","last_peered":"2024-04-25T01:01:38.551639+0000","last_clean":"2024-04-25T01:01:38.551639+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:01:38.551639+0000","last_undegraded":"2024-04-25T01:01:38.551639+0000","last_fullsized":"2024-04-25T01:01:38.551639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552136+0000","last_change":"2024-04-25T01:01:37.436247+0000","last_active":"2024-04-25T01:01:38.552136+0000","last_peered":"2024-04-25T01:01:38.552136+0000","last_clean":"2024-04-25T01:01:38.552136+0000","last_became_active":"2024-04-25T01:01:37.435388+0000","last_became_peered":"2024-04-25T01:01:37.435388+0000","last_unstale":"2024-04-25T01:01:38.552136+0000","last_undegraded":"2024-04-25T01:01:38.552136+0000","last_fullsized":"2024-04-25T01:01:38.552136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416152+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:01:38.416152+0000","last_peered":"2024-04-25T01:01:38.416152+0000","last_clean":"2024-04-25T01:01:38.416152+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:01:38.416152+0000","last_undegraded":"2024-04-25T01:01:38.416152+0000","last_fullsized":"2024-04-25T01:01:38.416152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415346+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:01:38.415346+0000","last_peered":"2024-04-25T01:01:38.415346+0000","last_clean":"2024-04-25T01:01:38.415346+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:01:38.415346+0000","last_undegraded":"2024-04-25T01:01:38.415346+0000","last_fullsized":"2024-04-25T01:01:38.415346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416144+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:01:38.416144+0000","last_peered":"2024-04-25T01:01:38.416144+0000","last_clean":"2024-04-25T01:01:38.416144+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:01:38.416144+0000","last_undegraded":"2024-04-25T01:01:38.416144+0000","last_fullsized":"2024-04-25T01:01:38.416144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468858+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:01:39.468858+0000","last_peered":"2024-04-25T01:01:39.468858+0000","last_clean":"2024-04-25T01:01:39.468858+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:01:39.468858+0000","last_undegraded":"2024-04-25T01:01:39.468858+0000","last_fullsized":"2024-04-25T01:01:39.468858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098463+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:01:39.098463+0000","last_peered":"2024-04-25T01:01:39.098463+0000","last_clean":"2024-04-25T01:01:39.098463+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:01:39.098463+0000","last_undegraded":"2024-04-25T01:01:39.098463+0000","last_fullsized":"2024-04-25T01:01:39.098463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415860+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:01:38.415860+0000","last_peered":"2024-04-25T01:01:38.415860+0000","last_clean":"2024-04-25T01:01:38.415860+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:01:38.415860+0000","last_undegraded":"2024-04-25T01:01:38.415860+0000","last_fullsized":"2024-04-25T01:01:38.415860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415706+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:01:38.415706+0000","last_peered":"2024-04-25T01:01:38.415706+0000","last_clean":"2024-04-25T01:01:38.415706+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:01:38.415706+0000","last_undegraded":"2024-04-25T01:01:38.415706+0000","last_fullsized":"2024-04-25T01:01:38.415706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552159+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:01:38.552159+0000","last_peered":"2024-04-25T01:01:38.552159+0000","last_clean":"2024-04-25T01:01:38.552159+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:01:38.552159+0000","last_undegraded":"2024-04-25T01:01:38.552159+0000","last_fullsized":"2024-04-25T01:01:38.552159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414731+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:01:38.414731+0000","last_peered":"2024-04-25T01:01:38.414731+0000","last_clean":"2024-04-25T01:01:38.414731+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:01:38.414731+0000","last_undegraded":"2024-04-25T01:01:38.414731+0000","last_fullsized":"2024-04-25T01:01:38.414731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552149+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:01:38.552149+0000","last_peered":"2024-04-25T01:01:38.552149+0000","last_clean":"2024-04-25T01:01:38.552149+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:01:38.552149+0000","last_undegraded":"2024-04-25T01:01:38.552149+0000","last_fullsized":"2024-04-25T01:01:38.552149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417245+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:01:38.417245+0000","last_peered":"2024-04-25T01:01:38.417245+0000","last_clean":"2024-04-25T01:01:38.417245+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:01:38.417245+0000","last_undegraded":"2024-04-25T01:01:38.417245+0000","last_fullsized":"2024-04-25T01:01:38.417245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468995+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:01:39.468995+0000","last_peered":"2024-04-25T01:01:39.468995+0000","last_clean":"2024-04-25T01:01:39.468995+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:01:39.468995+0000","last_undegraded":"2024-04-25T01:01:39.468995+0000","last_fullsized":"2024-04-25T01:01:39.468995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459566+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:01:38.459566+0000","last_peered":"2024-04-25T01:01:38.459566+0000","last_clean":"2024-04-25T01:01:38.459566+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:01:38.459566+0000","last_undegraded":"2024-04-25T01:01:38.459566+0000","last_fullsized":"2024-04-25T01:01:38.459566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552153+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:01:38.552153+0000","last_peered":"2024-04-25T01:01:38.552153+0000","last_clean":"2024-04-25T01:01:38.552153+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:01:38.552153+0000","last_undegraded":"2024-04-25T01:01:38.552153+0000","last_fullsized":"2024-04-25T01:01:38.552153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551709+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:01:38.551709+0000","last_peered":"2024-04-25T01:01:38.551709+0000","last_clean":"2024-04-25T01:01:38.551709+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:01:38.551709+0000","last_undegraded":"2024-04-25T01:01:38.551709+0000","last_fullsized":"2024-04-25T01:01:38.551709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416998+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:01:38.416998+0000","last_peered":"2024-04-25T01:01:38.416998+0000","last_clean":"2024-04-25T01:01:38.416998+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:01:38.416998+0000","last_undegraded":"2024-04-25T01:01:38.416998+0000","last_fullsized":"2024-04-25T01:01:38.416998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459358+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:01:38.459358+0000","last_peered":"2024-04-25T01:01:38.459358+0000","last_clean":"2024-04-25T01:01:38.459358+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:01:38.459358+0000","last_undegraded":"2024-04-25T01:01:38.459358+0000","last_fullsized":"2024-04-25T01:01:38.459358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552085+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:01:38.552085+0000","last_peered":"2024-04-25T01:01:38.552085+0000","last_clean":"2024-04-25T01:01:38.552085+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:01:38.552085+0000","last_undegraded":"2024-04-25T01:01:38.552085+0000","last_fullsized":"2024-04-25T01:01:38.552085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551641+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:01:38.551641+0000","last_peered":"2024-04-25T01:01:38.551641+0000","last_clean":"2024-04-25T01:01:38.551641+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:01:38.551641+0000","last_undegraded":"2024-04-25T01:01:38.551641+0000","last_fullsized":"2024-04-25T01:01:38.551641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575958+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:01:39.575958+0000","last_peered":"2024-04-25T01:01:39.575958+0000","last_clean":"2024-04-25T01:01:39.575958+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:01:39.575958+0000","last_undegraded":"2024-04-25T01:01:39.575958+0000","last_fullsized":"2024-04-25T01:01:39.575958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552170+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:01:38.552170+0000","last_peered":"2024-04-25T01:01:38.552170+0000","last_clean":"2024-04-25T01:01:38.552170+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:01:38.552170+0000","last_undegraded":"2024-04-25T01:01:38.552170+0000","last_fullsized":"2024-04-25T01:01:38.552170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468365+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:01:39.468365+0000","last_peered":"2024-04-25T01:01:39.468365+0000","last_clean":"2024-04-25T01:01:39.468365+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:01:39.468365+0000","last_undegraded":"2024-04-25T01:01:39.468365+0000","last_fullsized":"2024-04-25T01:01:39.468365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415600+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:01:38.415600+0000","last_peered":"2024-04-25T01:01:38.415600+0000","last_clean":"2024-04-25T01:01:38.415600+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:01:38.415600+0000","last_undegraded":"2024-04-25T01:01:38.415600+0000","last_fullsized":"2024-04-25T01:01:38.415600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468734+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:01:39.468734+0000","last_peered":"2024-04-25T01:01:39.468734+0000","last_clean":"2024-04-25T01:01:39.468734+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:01:39.468734+0000","last_undegraded":"2024-04-25T01:01:39.468734+0000","last_fullsized":"2024-04-25T01:01:39.468734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414950+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:01:38.414950+0000","last_peered":"2024-04-25T01:01:38.414950+0000","last_clean":"2024-04-25T01:01:38.414950+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:01:38.414950+0000","last_undegraded":"2024-04-25T01:01:38.414950+0000","last_fullsized":"2024-04-25T01:01:38.414950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459194+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:01:38.459194+0000","last_peered":"2024-04-25T01:01:38.459194+0000","last_clean":"2024-04-25T01:01:38.459194+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:01:38.459194+0000","last_undegraded":"2024-04-25T01:01:38.459194+0000","last_fullsized":"2024-04-25T01:01:38.459194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415719+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:01:38.415719+0000","last_peered":"2024-04-25T01:01:38.415719+0000","last_clean":"2024-04-25T01:01:38.415719+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:01:38.415719+0000","last_undegraded":"2024-04-25T01:01:38.415719+0000","last_fullsized":"2024-04-25T01:01:38.415719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098131+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:01:39.098131+0000","last_peered":"2024-04-25T01:01:39.098131+0000","last_clean":"2024-04-25T01:01:39.098131+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:01:39.098131+0000","last_undegraded":"2024-04-25T01:01:39.098131+0000","last_fullsized":"2024-04-25T01:01:39.098131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417023+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:01:38.417023+0000","last_peered":"2024-04-25T01:01:38.417023+0000","last_clean":"2024-04-25T01:01:38.417023+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:01:38.417023+0000","last_undegraded":"2024-04-25T01:01:38.417023+0000","last_fullsized":"2024-04-25T01:01:38.417023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576180+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:01:39.576180+0000","last_peered":"2024-04-25T01:01:39.576180+0000","last_clean":"2024-04-25T01:01:39.576180+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:01:39.576180+0000","last_undegraded":"2024-04-25T01:01:39.576180+0000","last_fullsized":"2024-04-25T01:01:39.576180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097822+0000","last_change":"2024-04-25T01:01:37.432081+0000","last_active":"2024-04-25T01:01:39.097822+0000","last_peered":"2024-04-25T01:01:39.097822+0000","last_clean":"2024-04-25T01:01:39.097822+0000","last_became_active":"2024-04-25T01:01:37.431657+0000","last_became_peered":"2024-04-25T01:01:37.431657+0000","last_unstale":"2024-04-25T01:01:39.097822+0000","last_undegraded":"2024-04-25T01:01:39.097822+0000","last_fullsized":"2024-04-25T01:01:39.097822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459452+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:01:38.459452+0000","last_peered":"2024-04-25T01:01:38.459452+0000","last_clean":"2024-04-25T01:01:38.459452+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:01:38.459452+0000","last_undegraded":"2024-04-25T01:01:38.459452+0000","last_fullsized":"2024-04-25T01:01:38.459452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552164+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:01:38.552164+0000","last_peered":"2024-04-25T01:01:38.552164+0000","last_clean":"2024-04-25T01:01:38.552164+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:01:38.552164+0000","last_undegraded":"2024-04-25T01:01:38.552164+0000","last_fullsized":"2024-04-25T01:01:38.552164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468921+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:01:39.468921+0000","last_peered":"2024-04-25T01:01:39.468921+0000","last_clean":"2024-04-25T01:01:39.468921+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:01:39.468921+0000","last_undegraded":"2024-04-25T01:01:39.468921+0000","last_fullsized":"2024-04-25T01:01:39.468921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417383+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:01:38.417383+0000","last_peered":"2024-04-25T01:01:38.417383+0000","last_clean":"2024-04-25T01:01:38.417383+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:01:38.417383+0000","last_undegraded":"2024-04-25T01:01:38.417383+0000","last_fullsized":"2024-04-25T01:01:38.417383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576094+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:01:39.576094+0000","last_peered":"2024-04-25T01:01:39.576094+0000","last_clean":"2024-04-25T01:01:39.576094+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:01:39.576094+0000","last_undegraded":"2024-04-25T01:01:39.576094+0000","last_fullsized":"2024-04-25T01:01:39.576094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458996+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:01:38.458996+0000","last_peered":"2024-04-25T01:01:38.458996+0000","last_clean":"2024-04-25T01:01:38.458996+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:01:38.458996+0000","last_undegraded":"2024-04-25T01:01:38.458996+0000","last_fullsized":"2024-04-25T01:01:38.458996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416863+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:01:38.416863+0000","last_peered":"2024-04-25T01:01:38.416863+0000","last_clean":"2024-04-25T01:01:38.416863+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:01:38.416863+0000","last_undegraded":"2024-04-25T01:01:38.416863+0000","last_fullsized":"2024-04-25T01:01:38.416863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416428+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:01:38.416428+0000","last_peered":"2024-04-25T01:01:38.416428+0000","last_clean":"2024-04-25T01:01:38.416428+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:01:38.416428+0000","last_undegraded":"2024-04-25T01:01:38.416428+0000","last_fullsized":"2024-04-25T01:01:38.416428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415619+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:01:38.415619+0000","last_peered":"2024-04-25T01:01:38.415619+0000","last_clean":"2024-04-25T01:01:38.415619+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:01:38.415619+0000","last_undegraded":"2024-04-25T01:01:38.415619+0000","last_fullsized":"2024-04-25T01:01:38.415619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415805+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:01:38.415805+0000","last_peered":"2024-04-25T01:01:38.415805+0000","last_clean":"2024-04-25T01:01:38.415805+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:01:38.415805+0000","last_undegraded":"2024-04-25T01:01:38.415805+0000","last_fullsized":"2024-04-25T01:01:38.415805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575808+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:01:39.575808+0000","last_peered":"2024-04-25T01:01:39.575808+0000","last_clean":"2024-04-25T01:01:39.575808+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:01:39.575808+0000","last_undegraded":"2024-04-25T01:01:39.575808+0000","last_fullsized":"2024-04-25T01:01:39.575808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468765+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:01:39.468765+0000","last_peered":"2024-04-25T01:01:39.468765+0000","last_clean":"2024-04-25T01:01:39.468765+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:01:39.468765+0000","last_undegraded":"2024-04-25T01:01:39.468765+0000","last_fullsized":"2024-04-25T01:01:39.468765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097771+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:01:39.097771+0000","last_peered":"2024-04-25T01:01:39.097771+0000","last_clean":"2024-04-25T01:01:39.097771+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:01:39.097771+0000","last_undegraded":"2024-04-25T01:01:39.097771+0000","last_fullsized":"2024-04-25T01:01:39.097771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097883+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:01:39.097883+0000","last_peered":"2024-04-25T01:01:39.097883+0000","last_clean":"2024-04-25T01:01:39.097883+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:01:39.097883+0000","last_undegraded":"2024-04-25T01:01:39.097883+0000","last_fullsized":"2024-04-25T01:01:39.097883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415652+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:01:38.415652+0000","last_peered":"2024-04-25T01:01:38.415652+0000","last_clean":"2024-04-25T01:01:38.415652+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:01:38.415652+0000","last_undegraded":"2024-04-25T01:01:38.415652+0000","last_fullsized":"2024-04-25T01:01:38.415652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416147+0000","last_change":"2024-04-25T01:01:33.389402+0000","last_active":"2024-04-25T01:01:38.416147+0000","last_peered":"2024-04-25T01:01:38.416147+0000","last_clean":"2024-04-25T01:01:38.416147+0000","last_became_active":"2024-04-25T01:01:33.389147+0000","last_became_peered":"2024-04-25T01:01:33.389147+0000","last_unstale":"2024-04-25T01:01:38.416147+0000","last_undegraded":"2024-04-25T01:01:38.416147+0000","last_fullsized":"2024-04-25T01:01:38.416147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459311+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:01:38.459311+0000","last_peered":"2024-04-25T01:01:38.459311+0000","last_clean":"2024-04-25T01:01:38.459311+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:01:38.459311+0000","last_undegraded":"2024-04-25T01:01:38.459311+0000","last_fullsized":"2024-04-25T01:01:38.459311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415085+0000","last_change":"2024-04-25T01:01:37.441477+0000","last_active":"2024-04-25T01:01:38.415085+0000","last_peered":"2024-04-25T01:01:38.415085+0000","last_clean":"2024-04-25T01:01:38.415085+0000","last_became_active":"2024-04-25T01:01:37.441099+0000","last_became_peered":"2024-04-25T01:01:37.441099+0000","last_unstale":"2024-04-25T01:01:38.415085+0000","last_undegraded":"2024-04-25T01:01:38.415085+0000","last_fullsized":"2024-04-25T01:01:38.415085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468846+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:01:39.468846+0000","last_peered":"2024-04-25T01:01:39.468846+0000","last_clean":"2024-04-25T01:01:39.468846+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:01:39.468846+0000","last_undegraded":"2024-04-25T01:01:39.468846+0000","last_fullsized":"2024-04-25T01:01:39.468846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469030+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:01:39.469030+0000","last_peered":"2024-04-25T01:01:39.469030+0000","last_clean":"2024-04-25T01:01:39.469030+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:01:39.469030+0000","last_undegraded":"2024-04-25T01:01:39.469030+0000","last_fullsized":"2024-04-25T01:01:39.469030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459574+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:01:38.459574+0000","last_peered":"2024-04-25T01:01:38.459574+0000","last_clean":"2024-04-25T01:01:38.459574+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:01:38.459574+0000","last_undegraded":"2024-04-25T01:01:38.459574+0000","last_fullsized":"2024-04-25T01:01:38.459574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416957+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:01:38.416957+0000","last_peered":"2024-04-25T01:01:38.416957+0000","last_clean":"2024-04-25T01:01:38.416957+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:01:38.416957+0000","last_undegraded":"2024-04-25T01:01:38.416957+0000","last_fullsized":"2024-04-25T01:01:38.416957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:43.083494+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:01:43.083494+0000","last_peered":"2024-04-25T01:01:43.083494+0000","last_clean":"2024-04-25T01:01:43.083494+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:01:43.083494+0000","last_undegraded":"2024-04-25T01:01:43.083494+0000","last_fullsized":"2024-04-25T01:01:43.083494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551940+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:01:38.551940+0000","last_peered":"2024-04-25T01:01:38.551940+0000","last_clean":"2024-04-25T01:01:38.551940+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:01:38.551940+0000","last_undegraded":"2024-04-25T01:01:38.551940+0000","last_fullsized":"2024-04-25T01:01:38.551940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416242+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:01:38.416242+0000","last_peered":"2024-04-25T01:01:38.416242+0000","last_clean":"2024-04-25T01:01:38.416242+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:01:38.416242+0000","last_undegraded":"2024-04-25T01:01:38.416242+0000","last_fullsized":"2024-04-25T01:01:38.416242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552062+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:01:38.552062+0000","last_peered":"2024-04-25T01:01:38.552062+0000","last_clean":"2024-04-25T01:01:38.552062+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:01:38.552062+0000","last_undegraded":"2024-04-25T01:01:38.552062+0000","last_fullsized":"2024-04-25T01:01:38.552062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416785+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:01:38.416785+0000","last_peered":"2024-04-25T01:01:38.416785+0000","last_clean":"2024-04-25T01:01:38.416785+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:01:38.416785+0000","last_undegraded":"2024-04-25T01:01:38.416785+0000","last_fullsized":"2024-04-25T01:01:38.416785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468828+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:01:39.468828+0000","last_peered":"2024-04-25T01:01:39.468828+0000","last_clean":"2024-04-25T01:01:39.468828+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:01:39.468828+0000","last_undegraded":"2024-04-25T01:01:39.468828+0000","last_fullsized":"2024-04-25T01:01:39.468828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459534+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:01:38.459534+0000","last_peered":"2024-04-25T01:01:38.459534+0000","last_clean":"2024-04-25T01:01:38.459534+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:01:38.459534+0000","last_undegraded":"2024-04-25T01:01:38.459534+0000","last_fullsized":"2024-04-25T01:01:38.459534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459526+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:01:38.459526+0000","last_peered":"2024-04-25T01:01:38.459526+0000","last_clean":"2024-04-25T01:01:38.459526+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:01:38.459526+0000","last_undegraded":"2024-04-25T01:01:38.459526+0000","last_fullsized":"2024-04-25T01:01:38.459526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414973+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:01:38.414973+0000","last_peered":"2024-04-25T01:01:38.414973+0000","last_clean":"2024-04-25T01:01:38.414973+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:01:38.414973+0000","last_undegraded":"2024-04-25T01:01:38.414973+0000","last_fullsized":"2024-04-25T01:01:38.414973+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468563+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:01:39.468563+0000","last_peered":"2024-04-25T01:01:39.468563+0000","last_clean":"2024-04-25T01:01:39.468563+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:01:39.468563+0000","last_undegraded":"2024-04-25T01:01:39.468563+0000","last_fullsized":"2024-04-25T01:01:39.468563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551523+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:01:38.551523+0000","last_peered":"2024-04-25T01:01:38.551523+0000","last_clean":"2024-04-25T01:01:38.551523+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:01:38.551523+0000","last_undegraded":"2024-04-25T01:01:38.551523+0000","last_fullsized":"2024-04-25T01:01:38.551523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459092+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:01:38.459092+0000","last_peered":"2024-04-25T01:01:38.459092+0000","last_clean":"2024-04-25T01:01:38.459092+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:01:38.459092+0000","last_undegraded":"2024-04-25T01:01:38.459092+0000","last_fullsized":"2024-04-25T01:01:38.459092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098352+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:01:39.098352+0000","last_peered":"2024-04-25T01:01:39.098352+0000","last_clean":"2024-04-25T01:01:39.098352+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:01:39.098352+0000","last_undegraded":"2024-04-25T01:01:39.098352+0000","last_fullsized":"2024-04-25T01:01:39.098352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414947+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:01:38.414947+0000","last_peered":"2024-04-25T01:01:38.414947+0000","last_clean":"2024-04-25T01:01:38.414947+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:01:38.414947+0000","last_undegraded":"2024-04-25T01:01:38.414947+0000","last_fullsized":"2024-04-25T01:01:38.414947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415843+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:01:38.415843+0000","last_peered":"2024-04-25T01:01:38.415843+0000","last_clean":"2024-04-25T01:01:38.415843+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:01:38.415843+0000","last_undegraded":"2024-04-25T01:01:38.415843+0000","last_fullsized":"2024-04-25T01:01:38.415843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097915+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:01:39.097915+0000","last_peered":"2024-04-25T01:01:39.097915+0000","last_clean":"2024-04-25T01:01:39.097915+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:01:39.097915+0000","last_undegraded":"2024-04-25T01:01:39.097915+0000","last_fullsized":"2024-04-25T01:01:39.097915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469001+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:01:39.469001+0000","last_peered":"2024-04-25T01:01:39.469001+0000","last_clean":"2024-04-25T01:01:39.469001+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:01:39.469001+0000","last_undegraded":"2024-04-25T01:01:39.469001+0000","last_fullsized":"2024-04-25T01:01:39.469001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576127+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:01:39.576127+0000","last_peered":"2024-04-25T01:01:39.576127+0000","last_clean":"2024-04-25T01:01:39.576127+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:01:39.576127+0000","last_undegraded":"2024-04-25T01:01:39.576127+0000","last_fullsized":"2024-04-25T01:01:39.576127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415723+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:01:38.415723+0000","last_peered":"2024-04-25T01:01:38.415723+0000","last_clean":"2024-04-25T01:01:38.415723+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:01:38.415723+0000","last_undegraded":"2024-04-25T01:01:38.415723+0000","last_fullsized":"2024-04-25T01:01:38.415723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416424+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:01:38.416424+0000","last_peered":"2024-04-25T01:01:38.416424+0000","last_clean":"2024-04-25T01:01:38.416424+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:01:38.416424+0000","last_undegraded":"2024-04-25T01:01:38.416424+0000","last_fullsized":"2024-04-25T01:01:38.416424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415424+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:01:38.415424+0000","last_peered":"2024-04-25T01:01:38.415424+0000","last_clean":"2024-04-25T01:01:38.415424+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:01:38.415424+0000","last_undegraded":"2024-04-25T01:01:38.415424+0000","last_fullsized":"2024-04-25T01:01:38.415424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097606+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:01:39.097606+0000","last_peered":"2024-04-25T01:01:39.097606+0000","last_clean":"2024-04-25T01:01:39.097606+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:01:39.097606+0000","last_undegraded":"2024-04-25T01:01:39.097606+0000","last_fullsized":"2024-04-25T01:01:39.097606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575834+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:01:39.575834+0000","last_peered":"2024-04-25T01:01:39.575834+0000","last_clean":"2024-04-25T01:01:39.575834+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:01:39.575834+0000","last_undegraded":"2024-04-25T01:01:39.575834+0000","last_fullsized":"2024-04-25T01:01:39.575834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459195+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:01:38.459195+0000","last_peered":"2024-04-25T01:01:38.459195+0000","last_clean":"2024-04-25T01:01:38.459195+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:01:38.459195+0000","last_undegraded":"2024-04-25T01:01:38.459195+0000","last_fullsized":"2024-04-25T01:01:38.459195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468951+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:01:39.468951+0000","last_peered":"2024-04-25T01:01:39.468951+0000","last_clean":"2024-04-25T01:01:39.468951+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:01:39.468951+0000","last_undegraded":"2024-04-25T01:01:39.468951+0000","last_fullsized":"2024-04-25T01:01:39.468951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415256+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:01:38.415256+0000","last_peered":"2024-04-25T01:01:38.415256+0000","last_clean":"2024-04-25T01:01:38.415256+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:01:38.415256+0000","last_undegraded":"2024-04-25T01:01:38.415256+0000","last_fullsized":"2024-04-25T01:01:38.415256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552244+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:01:38.552244+0000","last_peered":"2024-04-25T01:01:38.552244+0000","last_clean":"2024-04-25T01:01:38.552244+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:01:38.552244+0000","last_undegraded":"2024-04-25T01:01:38.552244+0000","last_fullsized":"2024-04-25T01:01:38.552244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415802+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:01:38.415802+0000","last_peered":"2024-04-25T01:01:38.415802+0000","last_clean":"2024-04-25T01:01:38.415802+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:01:38.415802+0000","last_undegraded":"2024-04-25T01:01:38.415802+0000","last_fullsized":"2024-04-25T01:01:38.415802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459070+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:01:38.459070+0000","last_peered":"2024-04-25T01:01:38.459070+0000","last_clean":"2024-04-25T01:01:38.459070+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:01:38.459070+0000","last_undegraded":"2024-04-25T01:01:38.459070+0000","last_fullsized":"2024-04-25T01:01:38.459070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415674+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:01:38.415674+0000","last_peered":"2024-04-25T01:01:38.415674+0000","last_clean":"2024-04-25T01:01:38.415674+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:01:38.415674+0000","last_undegraded":"2024-04-25T01:01:38.415674+0000","last_fullsized":"2024-04-25T01:01:38.415674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459211+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:01:38.459211+0000","last_peered":"2024-04-25T01:01:38.459211+0000","last_clean":"2024-04-25T01:01:38.459211+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:01:38.459211+0000","last_undegraded":"2024-04-25T01:01:38.459211+0000","last_fullsized":"2024-04-25T01:01:38.459211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468702+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:01:39.468702+0000","last_peered":"2024-04-25T01:01:39.468702+0000","last_clean":"2024-04-25T01:01:39.468702+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:01:39.468702+0000","last_undegraded":"2024-04-25T01:01:39.468702+0000","last_fullsized":"2024-04-25T01:01:39.468702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098040+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:01:39.098040+0000","last_peered":"2024-04-25T01:01:39.098040+0000","last_clean":"2024-04-25T01:01:39.098040+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:01:39.098040+0000","last_undegraded":"2024-04-25T01:01:39.098040+0000","last_fullsized":"2024-04-25T01:01:39.098040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417195+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:01:38.417195+0000","last_peered":"2024-04-25T01:01:38.417195+0000","last_clean":"2024-04-25T01:01:38.417195+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:01:38.417195+0000","last_undegraded":"2024-04-25T01:01:38.417195+0000","last_fullsized":"2024-04-25T01:01:38.417195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576085+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:01:39.576085+0000","last_peered":"2024-04-25T01:01:39.576085+0000","last_clean":"2024-04-25T01:01:39.576085+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:01:39.576085+0000","last_undegraded":"2024-04-25T01:01:39.576085+0000","last_fullsized":"2024-04-25T01:01:39.576085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459745+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:01:38.459745+0000","last_peered":"2024-04-25T01:01:38.459745+0000","last_clean":"2024-04-25T01:01:38.459745+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:01:38.459745+0000","last_undegraded":"2024-04-25T01:01:38.459745+0000","last_fullsized":"2024-04-25T01:01:38.459745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458917+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:01:38.458917+0000","last_peered":"2024-04-25T01:01:38.458917+0000","last_clean":"2024-04-25T01:01:38.458917+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:01:38.458917+0000","last_undegraded":"2024-04-25T01:01:38.458917+0000","last_fullsized":"2024-04-25T01:01:38.458917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551583+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:01:38.551583+0000","last_peered":"2024-04-25T01:01:38.551583+0000","last_clean":"2024-04-25T01:01:38.551583+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:01:38.551583+0000","last_undegraded":"2024-04-25T01:01:38.551583+0000","last_fullsized":"2024-04-25T01:01:38.551583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468688+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:01:39.468688+0000","last_peered":"2024-04-25T01:01:39.468688+0000","last_clean":"2024-04-25T01:01:39.468688+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:01:39.468688+0000","last_undegraded":"2024-04-25T01:01:39.468688+0000","last_fullsized":"2024-04-25T01:01:39.468688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415410+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:01:38.415410+0000","last_peered":"2024-04-25T01:01:38.415410+0000","last_clean":"2024-04-25T01:01:38.415410+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:01:38.415410+0000","last_undegraded":"2024-04-25T01:01:38.415410+0000","last_fullsized":"2024-04-25T01:01:38.415410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416131+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:01:38.416131+0000","last_peered":"2024-04-25T01:01:38.416131+0000","last_clean":"2024-04-25T01:01:38.416131+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:01:38.416131+0000","last_undegraded":"2024-04-25T01:01:38.416131+0000","last_fullsized":"2024-04-25T01:01:38.416131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098109+0000","last_change":"2024-04-25T01:01:33.405682+0000","last_active":"2024-04-25T01:01:39.098109+0000","last_peered":"2024-04-25T01:01:39.098109+0000","last_clean":"2024-04-25T01:01:39.098109+0000","last_became_active":"2024-04-25T01:01:33.405406+0000","last_became_peered":"2024-04-25T01:01:33.405406+0000","last_unstale":"2024-04-25T01:01:39.098109+0000","last_undegraded":"2024-04-25T01:01:39.098109+0000","last_fullsized":"2024-04-25T01:01:39.098109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098315+0000","last_change":"2024-04-25T01:01:33.414506+0000","last_active":"2024-04-25T01:01:39.098315+0000","last_peered":"2024-04-25T01:01:39.098315+0000","last_clean":"2024-04-25T01:01:39.098315+0000","last_became_active":"2024-04-25T01:01:33.414060+0000","last_became_peered":"2024-04-25T01:01:33.414060+0000","last_unstale":"2024-04-25T01:01:39.098315+0000","last_undegraded":"2024-04-25T01:01:39.098315+0000","last_fullsized":"2024-04-25T01:01:39.098315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417283+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:01:38.417283+0000","last_peered":"2024-04-25T01:01:38.417283+0000","last_clean":"2024-04-25T01:01:38.417283+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:01:38.417283+0000","last_undegraded":"2024-04-25T01:01:38.417283+0000","last_fullsized":"2024-04-25T01:01:38.417283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414797+0000","last_change":"2024-04-25T01:01:33.408326+0000","last_active":"2024-04-25T01:01:38.414797+0000","last_peered":"2024-04-25T01:01:38.414797+0000","last_clean":"2024-04-25T01:01:38.414797+0000","last_became_active":"2024-04-25T01:01:33.408039+0000","last_became_peered":"2024-04-25T01:01:33.408039+0000","last_unstale":"2024-04-25T01:01:38.414797+0000","last_undegraded":"2024-04-25T01:01:38.414797+0000","last_fullsized":"2024-04-25T01:01:38.414797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415697+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:01:38.415697+0000","last_peered":"2024-04-25T01:01:38.415697+0000","last_clean":"2024-04-25T01:01:38.415697+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:01:38.415697+0000","last_undegraded":"2024-04-25T01:01:38.415697+0000","last_fullsized":"2024-04-25T01:01:38.415697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459077+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:01:38.459077+0000","last_peered":"2024-04-25T01:01:38.459077+0000","last_clean":"2024-04-25T01:01:38.459077+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:01:38.459077+0000","last_undegraded":"2024-04-25T01:01:38.459077+0000","last_fullsized":"2024-04-25T01:01:38.459077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415902+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:01:38.415902+0000","last_peered":"2024-04-25T01:01:38.415902+0000","last_clean":"2024-04-25T01:01:38.415902+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:01:38.415902+0000","last_undegraded":"2024-04-25T01:01:38.415902+0000","last_fullsized":"2024-04-25T01:01:38.415902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459266+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:01:38.459266+0000","last_peered":"2024-04-25T01:01:38.459266+0000","last_clean":"2024-04-25T01:01:38.459266+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:01:38.459266+0000","last_undegraded":"2024-04-25T01:01:38.459266+0000","last_fullsized":"2024-04-25T01:01:38.459266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468558+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:01:39.468558+0000","last_peered":"2024-04-25T01:01:39.468558+0000","last_clean":"2024-04-25T01:01:39.468558+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:01:39.468558+0000","last_undegraded":"2024-04-25T01:01:39.468558+0000","last_fullsized":"2024-04-25T01:01:39.468558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552094+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:01:38.552094+0000","last_peered":"2024-04-25T01:01:38.552094+0000","last_clean":"2024-04-25T01:01:38.552094+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:01:38.552094+0000","last_undegraded":"2024-04-25T01:01:38.552094+0000","last_fullsized":"2024-04-25T01:01:38.552094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417169+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:01:38.417169+0000","last_peered":"2024-04-25T01:01:38.417169+0000","last_clean":"2024-04-25T01:01:38.417169+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:01:38.417169+0000","last_undegraded":"2024-04-25T01:01:38.417169+0000","last_fullsized":"2024-04-25T01:01:38.417169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:02:00.769 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:02:00.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.899+0000 7f09cfb90640 1 Processor -- start 2024-04-25T01:02:00.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.899+0000 7f09cfb90640 1 -- start start 2024-04-25T01:02:00.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cfb90640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f09c80c5520 0x7f09c80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cfb90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09c80c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cfb90640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c80c9860 con 0x7f09c8152fa0 2024-04-25T01:02:00.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cfb90640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f09c80c5a70 con 0x7f09c80c5520 2024-04-25T01:02:00.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cfb90640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f09c80c5d50 con 0x7f09c81561e0 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cd92c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f09c8152fa0 0x7f09c80c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44810/0 (socket says 172.21.15.129:44810) 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.900+0000 7f09cd92c640 1 -- 172.21.15.129:0/4051825579 learned_addr learned my addr 172.21.15.129:0/4051825579 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd12b640 1 --2- 172.21.15.129:0/4051825579 >> v2:172.21.15.171:3300/0 conn(0x7f09c80c5520 0x7f09c80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09ce12d640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09c80c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744655526 0 0) 0x7f09c4001490 con 0x7f09c8152fa0 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09b4003520 con 0x7f09c8152fa0 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2902751094 0 0) 0x7f09b4003520 con 0x7f09c8152fa0 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 msgr2=0x7f09c80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09c80c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 >> v2:172.21.15.171:3300/0 conn(0x7f09c80c5520 msgr2=0x7f09c80ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 --2- 172.21.15.129:0/4051825579 >> v2:172.21.15.171:3300/0 conn(0x7f09c80c5520 0x7f09c80ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c80c7650 con 0x7f09c8152fa0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f09c4001bc0 con 0x7f09c8152fa0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 >> v1:172.21.15.129:6789/0 conn(0x7f09c8152fa0 legacy=0x7f09c80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80c5a70 con 0x7f09b40039b0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80c5d50 con 0x7f09b4004800 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f09b400ba40 con 0x7f09c81561e0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd92c640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd12b640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09ce12d640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd92c640 1 -- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 msgr2=0x7f09b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd92c640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd92c640 1 -- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09b4004800 msgr2=0x7f09b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd92c640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd92c640 1 -- 172.21.15.129:0/4051825579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b4003520 con 0x7f09b40039b0 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09ce12d640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c81561e0 0x7f09b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.901+0000 7f09cd12b640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.902+0000 7f09cd92c640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f09c400aa40 tx=0x7f09b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e514821bb998c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.902+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c400a690 con 0x7f09b40039b0 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.902+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c402f070 con 0x7f09b40039b0 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.902+0000 7f09cc92a640 1 -- 172.21.15.129:0/4051825579 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4034470 con 0x7f09b40039b0 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09cfb90640 1 -- 172.21.15.129:0/4051825579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 msgr2=0x7f09b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09cfb90640 1 --2- 172.21.15.129:0/4051825579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f09c400aa40 tx=0x7f09b4005830 comp rx=0 tx=0).stop 2024-04-25T01:02:00.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09ce12d640 1 -- 172.21.15.129:0/4051825579 reap_dead start 2024-04-25T01:02:00.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09cfb90640 1 -- 172.21.15.129:0/4051825579 shutdown_connections 2024-04-25T01:02:00.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09cfb90640 1 -- 172.21.15.129:0/4051825579 >> 172.21.15.129:0/4051825579 conn(0x7f09c806c400 msgr2=0x7f09c806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:00.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09cfb90640 1 -- 172.21.15.129:0/4051825579 shutdown_connections 2024-04-25T01:02:00.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.903+0000 7f09cfb90640 1 -- 172.21.15.129:0/4051825579 wait complete. 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 Processor -- start 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 -- start start 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09c80bf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09c80c0710 0x7f09c8158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c8158940 0x7f09c80c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80c8cc0 con 0x7f09b40039b0 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80c6030 con 0x7f09c80c0710 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.904+0000 7f09cfb90640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f09c80c8e40 con 0x7f09c8158940 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09c80bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09c80bf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38808/0 (socket says 172.21.15.129:38808) 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 -- 172.21.15.129:0/1737603760 learned_addr learned my addr 172.21.15.129:0/1737603760 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd12b640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09c80c0710 0x7f09c8158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09ce12d640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c8158940 0x7f09c80c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 -- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c8158940 msgr2=0x7f09c80c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c8158940 0x7f09c80c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 -- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09c80c0710 msgr2=0x7f09c8158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09c80c0710 0x7f09c8158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 -- 172.21.15.129:0/1737603760 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c80bf560 con 0x7f09b40039b0 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09ce12d640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c8158940 0x7f09c80c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:00.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cd92c640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09c80bf020 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09c402b700 tx=0x7f09c8153380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c402ba50 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c402bbf0 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4034470 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c81ef280 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.905+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c80c4710 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.906+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c806b8a0 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.907+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09c4034610 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.907+0000 7f09be7fc640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f09ac06be60 0x7f09ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.907+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09c40ab3d0 con 0x7f09b40039b0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.907+0000 7f09cd12b640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f09ac06be60 0x7f09ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.908+0000 7f09cd12b640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f09ac06be60 0x7f09ac06e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09c80aae50 tx=0x7f09b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:00.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:00.911+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f09c806b8a0 con 0x7f09b40039b0 2024-04-25T01:02:01.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.078+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09c80ca0c0 con 0x7f09ac06be60 2024-04-25T01:02:01.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.086+0000 7f09be7fc640 1 -- 172.21.15.129:0/1737603760 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f09c80ca0c0 con 0x7f09ac06be60 2024-04-25T01:02:01.090 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:01.092 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f09ac06be60 msgr2=0x7f09ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f09ac06be60 0x7f09ac06e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09c80aae50 tx=0x7f09b8008040 comp rx=0 tx=0).stop 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 msgr2=0x7f09c80bf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09c80bf020 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09c402b700 tx=0x7f09c8153380 comp rx=0 tx=0).stop 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 shutdown_connections 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f09ac06be60 0x7f09ac06e320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09c8158940 0x7f09c80c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09c80c0710 0x7f09c8158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 --2- 172.21.15.129:0/1737603760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b40039b0 0x7f09c80bf020 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.089+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 >> 172.21.15.129:0/1737603760 conn(0x7f09c806c400 msgr2=0x7f09c806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.090+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 shutdown_connections 2024-04-25T01:02:01.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.090+0000 7f09cfb90640 1 -- 172.21.15.129:0/1737603760 wait complete. 2024-04-25T01:02:01.101 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T01:02:00.672306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219492,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755228,"statfs":{"total":773094113280,"available":772869353472,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003706"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468886+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:01:39.468886+0000","last_peered":"2024-04-25T01:01:39.468886+0000","last_clean":"2024-04-25T01:01:39.468886+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:01:39.468886+0000","last_undegraded":"2024-04-25T01:01:39.468886+0000","last_fullsized":"2024-04-25T01:01:39.468886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415373+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:01:38.415373+0000","last_peered":"2024-04-25T01:01:38.415373+0000","last_clean":"2024-04-25T01:01:38.415373+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:01:38.415373+0000","last_undegraded":"2024-04-25T01:01:38.415373+0000","last_fullsized":"2024-04-25T01:01:38.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414835+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:01:38.414835+0000","last_peered":"2024-04-25T01:01:38.414835+0000","last_clean":"2024-04-25T01:01:38.414835+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:01:38.414835+0000","last_undegraded":"2024-04-25T01:01:38.414835+0000","last_fullsized":"2024-04-25T01:01:38.414835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468718+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:01:39.468718+0000","last_peered":"2024-04-25T01:01:39.468718+0000","last_clean":"2024-04-25T01:01:39.468718+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:01:39.468718+0000","last_undegraded":"2024-04-25T01:01:39.468718+0000","last_fullsized":"2024-04-25T01:01:39.468718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415475+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:01:38.415475+0000","last_peered":"2024-04-25T01:01:38.415475+0000","last_clean":"2024-04-25T01:01:38.415475+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:01:38.415475+0000","last_undegraded":"2024-04-25T01:01:38.415475+0000","last_fullsized":"2024-04-25T01:01:38.415475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417116+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:01:38.417116+0000","last_peered":"2024-04-25T01:01:38.417116+0000","last_clean":"2024-04-25T01:01:38.417116+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:01:38.417116+0000","last_undegraded":"2024-04-25T01:01:38.417116+0000","last_fullsized":"2024-04-25T01:01:38.417116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459319+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:01:38.459319+0000","last_peered":"2024-04-25T01:01:38.459319+0000","last_clean":"2024-04-25T01:01:38.459319+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:01:38.459319+0000","last_undegraded":"2024-04-25T01:01:38.459319+0000","last_fullsized":"2024-04-25T01:01:38.459319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416961+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:01:38.416961+0000","last_peered":"2024-04-25T01:01:38.416961+0000","last_clean":"2024-04-25T01:01:38.416961+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:01:38.416961+0000","last_undegraded":"2024-04-25T01:01:38.416961+0000","last_fullsized":"2024-04-25T01:01:38.416961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416812+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:01:38.416812+0000","last_peered":"2024-04-25T01:01:38.416812+0000","last_clean":"2024-04-25T01:01:38.416812+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:01:38.416812+0000","last_undegraded":"2024-04-25T01:01:38.416812+0000","last_fullsized":"2024-04-25T01:01:38.416812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459435+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:01:38.459435+0000","last_peered":"2024-04-25T01:01:38.459435+0000","last_clean":"2024-04-25T01:01:38.459435+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:01:38.459435+0000","last_undegraded":"2024-04-25T01:01:38.459435+0000","last_fullsized":"2024-04-25T01:01:38.459435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416306+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:01:38.416306+0000","last_peered":"2024-04-25T01:01:38.416306+0000","last_clean":"2024-04-25T01:01:38.416306+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:01:38.416306+0000","last_undegraded":"2024-04-25T01:01:38.416306+0000","last_fullsized":"2024-04-25T01:01:38.416306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097726+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:01:39.097726+0000","last_peered":"2024-04-25T01:01:39.097726+0000","last_clean":"2024-04-25T01:01:39.097726+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:01:39.097726+0000","last_undegraded":"2024-04-25T01:01:39.097726+0000","last_fullsized":"2024-04-25T01:01:39.097726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416250+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:01:38.416250+0000","last_peered":"2024-04-25T01:01:38.416250+0000","last_clean":"2024-04-25T01:01:38.416250+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:01:38.416250+0000","last_undegraded":"2024-04-25T01:01:38.416250+0000","last_fullsized":"2024-04-25T01:01:38.416250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416610+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:01:38.416610+0000","last_peered":"2024-04-25T01:01:38.416610+0000","last_clean":"2024-04-25T01:01:38.416610+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:01:38.416610+0000","last_undegraded":"2024-04-25T01:01:38.416610+0000","last_fullsized":"2024-04-25T01:01:38.416610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415556+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:01:38.415556+0000","last_peered":"2024-04-25T01:01:38.415556+0000","last_clean":"2024-04-25T01:01:38.415556+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:01:38.415556+0000","last_undegraded":"2024-04-25T01:01:38.415556+0000","last_fullsized":"2024-04-25T01:01:38.415556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551986+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:01:38.551986+0000","last_peered":"2024-04-25T01:01:38.551986+0000","last_clean":"2024-04-25T01:01:38.551986+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:01:38.551986+0000","last_undegraded":"2024-04-25T01:01:38.551986+0000","last_fullsized":"2024-04-25T01:01:38.551986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415994+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:01:38.415994+0000","last_peered":"2024-04-25T01:01:38.415994+0000","last_clean":"2024-04-25T01:01:38.415994+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:01:38.415994+0000","last_undegraded":"2024-04-25T01:01:38.415994+0000","last_fullsized":"2024-04-25T01:01:38.415994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415087+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:01:38.415087+0000","last_peered":"2024-04-25T01:01:38.415087+0000","last_clean":"2024-04-25T01:01:38.415087+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:01:38.415087+0000","last_undegraded":"2024-04-25T01:01:38.415087+0000","last_fullsized":"2024-04-25T01:01:38.415087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551639+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:01:38.551639+0000","last_peered":"2024-04-25T01:01:38.551639+0000","last_clean":"2024-04-25T01:01:38.551639+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:01:38.551639+0000","last_undegraded":"2024-04-25T01:01:38.551639+0000","last_fullsized":"2024-04-25T01:01:38.551639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552136+0000","last_change":"2024-04-25T01:01:37.436247+0000","last_active":"2024-04-25T01:01:38.552136+0000","last_peered":"2024-04-25T01:01:38.552136+0000","last_clean":"2024-04-25T01:01:38.552136+0000","last_became_active":"2024-04-25T01:01:37.435388+0000","last_became_peered":"2024-04-25T01:01:37.435388+0000","last_unstale":"2024-04-25T01:01:38.552136+0000","last_undegraded":"2024-04-25T01:01:38.552136+0000","last_fullsized":"2024-04-25T01:01:38.552136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416152+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:01:38.416152+0000","last_peered":"2024-04-25T01:01:38.416152+0000","last_clean":"2024-04-25T01:01:38.416152+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:01:38.416152+0000","last_undegraded":"2024-04-25T01:01:38.416152+0000","last_fullsized":"2024-04-25T01:01:38.416152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415346+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:01:38.415346+0000","last_peered":"2024-04-25T01:01:38.415346+0000","last_clean":"2024-04-25T01:01:38.415346+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:01:38.415346+0000","last_undegraded":"2024-04-25T01:01:38.415346+0000","last_fullsized":"2024-04-25T01:01:38.415346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416144+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:01:38.416144+0000","last_peered":"2024-04-25T01:01:38.416144+0000","last_clean":"2024-04-25T01:01:38.416144+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:01:38.416144+0000","last_undegraded":"2024-04-25T01:01:38.416144+0000","last_fullsized":"2024-04-25T01:01:38.416144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468858+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:01:39.468858+0000","last_peered":"2024-04-25T01:01:39.468858+0000","last_clean":"2024-04-25T01:01:39.468858+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:01:39.468858+0000","last_undegraded":"2024-04-25T01:01:39.468858+0000","last_fullsized":"2024-04-25T01:01:39.468858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098463+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:01:39.098463+0000","last_peered":"2024-04-25T01:01:39.098463+0000","last_clean":"2024-04-25T01:01:39.098463+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:01:39.098463+0000","last_undegraded":"2024-04-25T01:01:39.098463+0000","last_fullsized":"2024-04-25T01:01:39.098463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415860+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:01:38.415860+0000","last_peered":"2024-04-25T01:01:38.415860+0000","last_clean":"2024-04-25T01:01:38.415860+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:01:38.415860+0000","last_undegraded":"2024-04-25T01:01:38.415860+0000","last_fullsized":"2024-04-25T01:01:38.415860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415706+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:01:38.415706+0000","last_peered":"2024-04-25T01:01:38.415706+0000","last_clean":"2024-04-25T01:01:38.415706+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:01:38.415706+0000","last_undegraded":"2024-04-25T01:01:38.415706+0000","last_fullsized":"2024-04-25T01:01:38.415706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552159+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:01:38.552159+0000","last_peered":"2024-04-25T01:01:38.552159+0000","last_clean":"2024-04-25T01:01:38.552159+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:01:38.552159+0000","last_undegraded":"2024-04-25T01:01:38.552159+0000","last_fullsized":"2024-04-25T01:01:38.552159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414731+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:01:38.414731+0000","last_peered":"2024-04-25T01:01:38.414731+0000","last_clean":"2024-04-25T01:01:38.414731+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:01:38.414731+0000","last_undegraded":"2024-04-25T01:01:38.414731+0000","last_fullsized":"2024-04-25T01:01:38.414731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552149+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:01:38.552149+0000","last_peered":"2024-04-25T01:01:38.552149+0000","last_clean":"2024-04-25T01:01:38.552149+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:01:38.552149+0000","last_undegraded":"2024-04-25T01:01:38.552149+0000","last_fullsized":"2024-04-25T01:01:38.552149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417245+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:01:38.417245+0000","last_peered":"2024-04-25T01:01:38.417245+0000","last_clean":"2024-04-25T01:01:38.417245+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:01:38.417245+0000","last_undegraded":"2024-04-25T01:01:38.417245+0000","last_fullsized":"2024-04-25T01:01:38.417245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468995+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:01:39.468995+0000","last_peered":"2024-04-25T01:01:39.468995+0000","last_clean":"2024-04-25T01:01:39.468995+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:01:39.468995+0000","last_undegraded":"2024-04-25T01:01:39.468995+0000","last_fullsized":"2024-04-25T01:01:39.468995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459566+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:01:38.459566+0000","last_peered":"2024-04-25T01:01:38.459566+0000","last_clean":"2024-04-25T01:01:38.459566+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:01:38.459566+0000","last_undegraded":"2024-04-25T01:01:38.459566+0000","last_fullsized":"2024-04-25T01:01:38.459566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552153+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:01:38.552153+0000","last_peered":"2024-04-25T01:01:38.552153+0000","last_clean":"2024-04-25T01:01:38.552153+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:01:38.552153+0000","last_undegraded":"2024-04-25T01:01:38.552153+0000","last_fullsized":"2024-04-25T01:01:38.552153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551709+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:01:38.551709+0000","last_peered":"2024-04-25T01:01:38.551709+0000","last_clean":"2024-04-25T01:01:38.551709+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:01:38.551709+0000","last_undegraded":"2024-04-25T01:01:38.551709+0000","last_fullsized":"2024-04-25T01:01:38.551709+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416998+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:01:38.416998+0000","last_peered":"2024-04-25T01:01:38.416998+0000","last_clean":"2024-04-25T01:01:38.416998+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:01:38.416998+0000","last_undegraded":"2024-04-25T01:01:38.416998+0000","last_fullsized":"2024-04-25T01:01:38.416998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459358+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:01:38.459358+0000","last_peered":"2024-04-25T01:01:38.459358+0000","last_clean":"2024-04-25T01:01:38.459358+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:01:38.459358+0000","last_undegraded":"2024-04-25T01:01:38.459358+0000","last_fullsized":"2024-04-25T01:01:38.459358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552085+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:01:38.552085+0000","last_peered":"2024-04-25T01:01:38.552085+0000","last_clean":"2024-04-25T01:01:38.552085+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:01:38.552085+0000","last_undegraded":"2024-04-25T01:01:38.552085+0000","last_fullsized":"2024-04-25T01:01:38.552085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551641+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:01:38.551641+0000","last_peered":"2024-04-25T01:01:38.551641+0000","last_clean":"2024-04-25T01:01:38.551641+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:01:38.551641+0000","last_undegraded":"2024-04-25T01:01:38.551641+0000","last_fullsized":"2024-04-25T01:01:38.551641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575958+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:01:39.575958+0000","last_peered":"2024-04-25T01:01:39.575958+0000","last_clean":"2024-04-25T01:01:39.575958+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:01:39.575958+0000","last_undegraded":"2024-04-25T01:01:39.575958+0000","last_fullsized":"2024-04-25T01:01:39.575958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552170+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:01:38.552170+0000","last_peered":"2024-04-25T01:01:38.552170+0000","last_clean":"2024-04-25T01:01:38.552170+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:01:38.552170+0000","last_undegraded":"2024-04-25T01:01:38.552170+0000","last_fullsized":"2024-04-25T01:01:38.552170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468365+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:01:39.468365+0000","last_peered":"2024-04-25T01:01:39.468365+0000","last_clean":"2024-04-25T01:01:39.468365+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:01:39.468365+0000","last_undegraded":"2024-04-25T01:01:39.468365+0000","last_fullsized":"2024-04-25T01:01:39.468365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415600+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:01:38.415600+0000","last_peered":"2024-04-25T01:01:38.415600+0000","last_clean":"2024-04-25T01:01:38.415600+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:01:38.415600+0000","last_undegraded":"2024-04-25T01:01:38.415600+0000","last_fullsized":"2024-04-25T01:01:38.415600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468734+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:01:39.468734+0000","last_peered":"2024-04-25T01:01:39.468734+0000","last_clean":"2024-04-25T01:01:39.468734+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:01:39.468734+0000","last_undegraded":"2024-04-25T01:01:39.468734+0000","last_fullsized":"2024-04-25T01:01:39.468734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414950+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:01:38.414950+0000","last_peered":"2024-04-25T01:01:38.414950+0000","last_clean":"2024-04-25T01:01:38.414950+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:01:38.414950+0000","last_undegraded":"2024-04-25T01:01:38.414950+0000","last_fullsized":"2024-04-25T01:01:38.414950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459194+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:01:38.459194+0000","last_peered":"2024-04-25T01:01:38.459194+0000","last_clean":"2024-04-25T01:01:38.459194+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:01:38.459194+0000","last_undegraded":"2024-04-25T01:01:38.459194+0000","last_fullsized":"2024-04-25T01:01:38.459194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415719+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:01:38.415719+0000","last_peered":"2024-04-25T01:01:38.415719+0000","last_clean":"2024-04-25T01:01:38.415719+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:01:38.415719+0000","last_undegraded":"2024-04-25T01:01:38.415719+0000","last_fullsized":"2024-04-25T01:01:38.415719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098131+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:01:39.098131+0000","last_peered":"2024-04-25T01:01:39.098131+0000","last_clean":"2024-04-25T01:01:39.098131+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:01:39.098131+0000","last_undegraded":"2024-04-25T01:01:39.098131+0000","last_fullsized":"2024-04-25T01:01:39.098131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417023+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:01:38.417023+0000","last_peered":"2024-04-25T01:01:38.417023+0000","last_clean":"2024-04-25T01:01:38.417023+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:01:38.417023+0000","last_undegraded":"2024-04-25T01:01:38.417023+0000","last_fullsized":"2024-04-25T01:01:38.417023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576180+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:01:39.576180+0000","last_peered":"2024-04-25T01:01:39.576180+0000","last_clean":"2024-04-25T01:01:39.576180+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:01:39.576180+0000","last_undegraded":"2024-04-25T01:01:39.576180+0000","last_fullsized":"2024-04-25T01:01:39.576180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097822+0000","last_change":"2024-04-25T01:01:37.432081+0000","last_active":"2024-04-25T01:01:39.097822+0000","last_peered":"2024-04-25T01:01:39.097822+0000","last_clean":"2024-04-25T01:01:39.097822+0000","last_became_active":"2024-04-25T01:01:37.431657+0000","last_became_peered":"2024-04-25T01:01:37.431657+0000","last_unstale":"2024-04-25T01:01:39.097822+0000","last_undegraded":"2024-04-25T01:01:39.097822+0000","last_fullsized":"2024-04-25T01:01:39.097822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459452+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:01:38.459452+0000","last_peered":"2024-04-25T01:01:38.459452+0000","last_clean":"2024-04-25T01:01:38.459452+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:01:38.459452+0000","last_undegraded":"2024-04-25T01:01:38.459452+0000","last_fullsized":"2024-04-25T01:01:38.459452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552164+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:01:38.552164+0000","last_peered":"2024-04-25T01:01:38.552164+0000","last_clean":"2024-04-25T01:01:38.552164+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:01:38.552164+0000","last_undegraded":"2024-04-25T01:01:38.552164+0000","last_fullsized":"2024-04-25T01:01:38.552164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468921+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:01:39.468921+0000","last_peered":"2024-04-25T01:01:39.468921+0000","last_clean":"2024-04-25T01:01:39.468921+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:01:39.468921+0000","last_undegraded":"2024-04-25T01:01:39.468921+0000","last_fullsized":"2024-04-25T01:01:39.468921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417383+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:01:38.417383+0000","last_peered":"2024-04-25T01:01:38.417383+0000","last_clean":"2024-04-25T01:01:38.417383+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:01:38.417383+0000","last_undegraded":"2024-04-25T01:01:38.417383+0000","last_fullsized":"2024-04-25T01:01:38.417383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576094+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:01:39.576094+0000","last_peered":"2024-04-25T01:01:39.576094+0000","last_clean":"2024-04-25T01:01:39.576094+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:01:39.576094+0000","last_undegraded":"2024-04-25T01:01:39.576094+0000","last_fullsized":"2024-04-25T01:01:39.576094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458996+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:01:38.458996+0000","last_peered":"2024-04-25T01:01:38.458996+0000","last_clean":"2024-04-25T01:01:38.458996+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:01:38.458996+0000","last_undegraded":"2024-04-25T01:01:38.458996+0000","last_fullsized":"2024-04-25T01:01:38.458996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416863+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:01:38.416863+0000","last_peered":"2024-04-25T01:01:38.416863+0000","last_clean":"2024-04-25T01:01:38.416863+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:01:38.416863+0000","last_undegraded":"2024-04-25T01:01:38.416863+0000","last_fullsized":"2024-04-25T01:01:38.416863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416428+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:01:38.416428+0000","last_peered":"2024-04-25T01:01:38.416428+0000","last_clean":"2024-04-25T01:01:38.416428+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:01:38.416428+0000","last_undegraded":"2024-04-25T01:01:38.416428+0000","last_fullsized":"2024-04-25T01:01:38.416428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415619+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:01:38.415619+0000","last_peered":"2024-04-25T01:01:38.415619+0000","last_clean":"2024-04-25T01:01:38.415619+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:01:38.415619+0000","last_undegraded":"2024-04-25T01:01:38.415619+0000","last_fullsized":"2024-04-25T01:01:38.415619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415805+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:01:38.415805+0000","last_peered":"2024-04-25T01:01:38.415805+0000","last_clean":"2024-04-25T01:01:38.415805+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:01:38.415805+0000","last_undegraded":"2024-04-25T01:01:38.415805+0000","last_fullsized":"2024-04-25T01:01:38.415805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575808+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:01:39.575808+0000","last_peered":"2024-04-25T01:01:39.575808+0000","last_clean":"2024-04-25T01:01:39.575808+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:01:39.575808+0000","last_undegraded":"2024-04-25T01:01:39.575808+0000","last_fullsized":"2024-04-25T01:01:39.575808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468765+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:01:39.468765+0000","last_peered":"2024-04-25T01:01:39.468765+0000","last_clean":"2024-04-25T01:01:39.468765+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:01:39.468765+0000","last_undegraded":"2024-04-25T01:01:39.468765+0000","last_fullsized":"2024-04-25T01:01:39.468765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097771+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:01:39.097771+0000","last_peered":"2024-04-25T01:01:39.097771+0000","last_clean":"2024-04-25T01:01:39.097771+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:01:39.097771+0000","last_undegraded":"2024-04-25T01:01:39.097771+0000","last_fullsized":"2024-04-25T01:01:39.097771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097883+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:01:39.097883+0000","last_peered":"2024-04-25T01:01:39.097883+0000","last_clean":"2024-04-25T01:01:39.097883+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:01:39.097883+0000","last_undegraded":"2024-04-25T01:01:39.097883+0000","last_fullsized":"2024-04-25T01:01:39.097883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415652+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:01:38.415652+0000","last_peered":"2024-04-25T01:01:38.415652+0000","last_clean":"2024-04-25T01:01:38.415652+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:01:38.415652+0000","last_undegraded":"2024-04-25T01:01:38.415652+0000","last_fullsized":"2024-04-25T01:01:38.415652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416147+0000","last_change":"2024-04-25T01:01:33.389402+0000","last_active":"2024-04-25T01:01:38.416147+0000","last_peered":"2024-04-25T01:01:38.416147+0000","last_clean":"2024-04-25T01:01:38.416147+0000","last_became_active":"2024-04-25T01:01:33.389147+0000","last_became_peered":"2024-04-25T01:01:33.389147+0000","last_unstale":"2024-04-25T01:01:38.416147+0000","last_undegraded":"2024-04-25T01:01:38.416147+0000","last_fullsized":"2024-04-25T01:01:38.416147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459311+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:01:38.459311+0000","last_peered":"2024-04-25T01:01:38.459311+0000","last_clean":"2024-04-25T01:01:38.459311+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:01:38.459311+0000","last_undegraded":"2024-04-25T01:01:38.459311+0000","last_fullsized":"2024-04-25T01:01:38.459311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415085+0000","last_change":"2024-04-25T01:01:37.441477+0000","last_active":"2024-04-25T01:01:38.415085+0000","last_peered":"2024-04-25T01:01:38.415085+0000","last_clean":"2024-04-25T01:01:38.415085+0000","last_became_active":"2024-04-25T01:01:37.441099+0000","last_became_peered":"2024-04-25T01:01:37.441099+0000","last_unstale":"2024-04-25T01:01:38.415085+0000","last_undegraded":"2024-04-25T01:01:38.415085+0000","last_fullsized":"2024-04-25T01:01:38.415085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468846+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:01:39.468846+0000","last_peered":"2024-04-25T01:01:39.468846+0000","last_clean":"2024-04-25T01:01:39.468846+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:01:39.468846+0000","last_undegraded":"2024-04-25T01:01:39.468846+0000","last_fullsized":"2024-04-25T01:01:39.468846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469030+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:01:39.469030+0000","last_peered":"2024-04-25T01:01:39.469030+0000","last_clean":"2024-04-25T01:01:39.469030+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:01:39.469030+0000","last_undegraded":"2024-04-25T01:01:39.469030+0000","last_fullsized":"2024-04-25T01:01:39.469030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459574+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:01:38.459574+0000","last_peered":"2024-04-25T01:01:38.459574+0000","last_clean":"2024-04-25T01:01:38.459574+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:01:38.459574+0000","last_undegraded":"2024-04-25T01:01:38.459574+0000","last_fullsized":"2024-04-25T01:01:38.459574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416957+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:01:38.416957+0000","last_peered":"2024-04-25T01:01:38.416957+0000","last_clean":"2024-04-25T01:01:38.416957+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:01:38.416957+0000","last_undegraded":"2024-04-25T01:01:38.416957+0000","last_fullsized":"2024-04-25T01:01:38.416957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:43.083494+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:01:43.083494+0000","last_peered":"2024-04-25T01:01:43.083494+0000","last_clean":"2024-04-25T01:01:43.083494+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:01:43.083494+0000","last_undegraded":"2024-04-25T01:01:43.083494+0000","last_fullsized":"2024-04-25T01:01:43.083494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551940+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:01:38.551940+0000","last_peered":"2024-04-25T01:01:38.551940+0000","last_clean":"2024-04-25T01:01:38.551940+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:01:38.551940+0000","last_undegraded":"2024-04-25T01:01:38.551940+0000","last_fullsized":"2024-04-25T01:01:38.551940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416242+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:01:38.416242+0000","last_peered":"2024-04-25T01:01:38.416242+0000","last_clean":"2024-04-25T01:01:38.416242+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:01:38.416242+0000","last_undegraded":"2024-04-25T01:01:38.416242+0000","last_fullsized":"2024-04-25T01:01:38.416242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552062+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:01:38.552062+0000","last_peered":"2024-04-25T01:01:38.552062+0000","last_clean":"2024-04-25T01:01:38.552062+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:01:38.552062+0000","last_undegraded":"2024-04-25T01:01:38.552062+0000","last_fullsized":"2024-04-25T01:01:38.552062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416785+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:01:38.416785+0000","last_peered":"2024-04-25T01:01:38.416785+0000","last_clean":"2024-04-25T01:01:38.416785+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:01:38.416785+0000","last_undegraded":"2024-04-25T01:01:38.416785+0000","last_fullsized":"2024-04-25T01:01:38.416785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468828+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:01:39.468828+0000","last_peered":"2024-04-25T01:01:39.468828+0000","last_clean":"2024-04-25T01:01:39.468828+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:01:39.468828+0000","last_undegraded":"2024-04-25T01:01:39.468828+0000","last_fullsized":"2024-04-25T01:01:39.468828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459534+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:01:38.459534+0000","last_peered":"2024-04-25T01:01:38.459534+0000","last_clean":"2024-04-25T01:01:38.459534+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:01:38.459534+0000","last_undegraded":"2024-04-25T01:01:38.459534+0000","last_fullsized":"2024-04-25T01:01:38.459534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415672+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:01:38.415672+0000","last_peered":"2024-04-25T01:01:38.415672+0000","last_clean":"2024-04-25T01:01:38.415672+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:01:38.415672+0000","last_undegraded":"2024-04-25T01:01:38.415672+0000","last_fullsized":"2024-04-25T01:01:38.415672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459526+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:01:38.459526+0000","last_peered":"2024-04-25T01:01:38.459526+0000","last_clean":"2024-04-25T01:01:38.459526+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:01:38.459526+0000","last_undegraded":"2024-04-25T01:01:38.459526+0000","last_fullsized":"2024-04-25T01:01:38.459526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414973+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:01:38.414973+0000","last_peered":"2024-04-25T01:01:38.414973+0000","last_clean":"2024-04-25T01:01:38.414973+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:01:38.414973+0000","last_undegraded":"2024-04-25T01:01:38.414973+0000","last_fullsized":"2024-04-25T01:01:38.414973+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468563+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:01:39.468563+0000","last_peered":"2024-04-25T01:01:39.468563+0000","last_clean":"2024-04-25T01:01:39.468563+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:01:39.468563+0000","last_undegraded":"2024-04-25T01:01:39.468563+0000","last_fullsized":"2024-04-25T01:01:39.468563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551523+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:01:38.551523+0000","last_peered":"2024-04-25T01:01:38.551523+0000","last_clean":"2024-04-25T01:01:38.551523+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:01:38.551523+0000","last_undegraded":"2024-04-25T01:01:38.551523+0000","last_fullsized":"2024-04-25T01:01:38.551523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459092+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:01:38.459092+0000","last_peered":"2024-04-25T01:01:38.459092+0000","last_clean":"2024-04-25T01:01:38.459092+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:01:38.459092+0000","last_undegraded":"2024-04-25T01:01:38.459092+0000","last_fullsized":"2024-04-25T01:01:38.459092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098352+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:01:39.098352+0000","last_peered":"2024-04-25T01:01:39.098352+0000","last_clean":"2024-04-25T01:01:39.098352+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:01:39.098352+0000","last_undegraded":"2024-04-25T01:01:39.098352+0000","last_fullsized":"2024-04-25T01:01:39.098352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414947+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:01:38.414947+0000","last_peered":"2024-04-25T01:01:38.414947+0000","last_clean":"2024-04-25T01:01:38.414947+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:01:38.414947+0000","last_undegraded":"2024-04-25T01:01:38.414947+0000","last_fullsized":"2024-04-25T01:01:38.414947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415843+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:01:38.415843+0000","last_peered":"2024-04-25T01:01:38.415843+0000","last_clean":"2024-04-25T01:01:38.415843+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:01:38.415843+0000","last_undegraded":"2024-04-25T01:01:38.415843+0000","last_fullsized":"2024-04-25T01:01:38.415843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097915+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:01:39.097915+0000","last_peered":"2024-04-25T01:01:39.097915+0000","last_clean":"2024-04-25T01:01:39.097915+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:01:39.097915+0000","last_undegraded":"2024-04-25T01:01:39.097915+0000","last_fullsized":"2024-04-25T01:01:39.097915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.469001+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:01:39.469001+0000","last_peered":"2024-04-25T01:01:39.469001+0000","last_clean":"2024-04-25T01:01:39.469001+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:01:39.469001+0000","last_undegraded":"2024-04-25T01:01:39.469001+0000","last_fullsized":"2024-04-25T01:01:39.469001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576127+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:01:39.576127+0000","last_peered":"2024-04-25T01:01:39.576127+0000","last_clean":"2024-04-25T01:01:39.576127+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:01:39.576127+0000","last_undegraded":"2024-04-25T01:01:39.576127+0000","last_fullsized":"2024-04-25T01:01:39.576127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415723+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:01:38.415723+0000","last_peered":"2024-04-25T01:01:38.415723+0000","last_clean":"2024-04-25T01:01:38.415723+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:01:38.415723+0000","last_undegraded":"2024-04-25T01:01:38.415723+0000","last_fullsized":"2024-04-25T01:01:38.415723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416424+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:01:38.416424+0000","last_peered":"2024-04-25T01:01:38.416424+0000","last_clean":"2024-04-25T01:01:38.416424+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:01:38.416424+0000","last_undegraded":"2024-04-25T01:01:38.416424+0000","last_fullsized":"2024-04-25T01:01:38.416424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415424+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:01:38.415424+0000","last_peered":"2024-04-25T01:01:38.415424+0000","last_clean":"2024-04-25T01:01:38.415424+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:01:38.415424+0000","last_undegraded":"2024-04-25T01:01:38.415424+0000","last_fullsized":"2024-04-25T01:01:38.415424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.097606+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:01:39.097606+0000","last_peered":"2024-04-25T01:01:39.097606+0000","last_clean":"2024-04-25T01:01:39.097606+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:01:39.097606+0000","last_undegraded":"2024-04-25T01:01:39.097606+0000","last_fullsized":"2024-04-25T01:01:39.097606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.575834+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:01:39.575834+0000","last_peered":"2024-04-25T01:01:39.575834+0000","last_clean":"2024-04-25T01:01:39.575834+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:01:39.575834+0000","last_undegraded":"2024-04-25T01:01:39.575834+0000","last_fullsized":"2024-04-25T01:01:39.575834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459195+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:01:38.459195+0000","last_peered":"2024-04-25T01:01:38.459195+0000","last_clean":"2024-04-25T01:01:38.459195+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:01:38.459195+0000","last_undegraded":"2024-04-25T01:01:38.459195+0000","last_fullsized":"2024-04-25T01:01:38.459195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468951+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:01:39.468951+0000","last_peered":"2024-04-25T01:01:39.468951+0000","last_clean":"2024-04-25T01:01:39.468951+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:01:39.468951+0000","last_undegraded":"2024-04-25T01:01:39.468951+0000","last_fullsized":"2024-04-25T01:01:39.468951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415256+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:01:38.415256+0000","last_peered":"2024-04-25T01:01:38.415256+0000","last_clean":"2024-04-25T01:01:38.415256+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:01:38.415256+0000","last_undegraded":"2024-04-25T01:01:38.415256+0000","last_fullsized":"2024-04-25T01:01:38.415256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552244+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:01:38.552244+0000","last_peered":"2024-04-25T01:01:38.552244+0000","last_clean":"2024-04-25T01:01:38.552244+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:01:38.552244+0000","last_undegraded":"2024-04-25T01:01:38.552244+0000","last_fullsized":"2024-04-25T01:01:38.552244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098377+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:01:39.098377+0000","last_peered":"2024-04-25T01:01:39.098377+0000","last_clean":"2024-04-25T01:01:39.098377+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:01:39.098377+0000","last_undegraded":"2024-04-25T01:01:39.098377+0000","last_fullsized":"2024-04-25T01:01:39.098377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415802+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:01:38.415802+0000","last_peered":"2024-04-25T01:01:38.415802+0000","last_clean":"2024-04-25T01:01:38.415802+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:01:38.415802+0000","last_undegraded":"2024-04-25T01:01:38.415802+0000","last_fullsized":"2024-04-25T01:01:38.415802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459070+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:01:38.459070+0000","last_peered":"2024-04-25T01:01:38.459070+0000","last_clean":"2024-04-25T01:01:38.459070+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:01:38.459070+0000","last_undegraded":"2024-04-25T01:01:38.459070+0000","last_fullsized":"2024-04-25T01:01:38.459070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415674+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:01:38.415674+0000","last_peered":"2024-04-25T01:01:38.415674+0000","last_clean":"2024-04-25T01:01:38.415674+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:01:38.415674+0000","last_undegraded":"2024-04-25T01:01:38.415674+0000","last_fullsized":"2024-04-25T01:01:38.415674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459211+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:01:38.459211+0000","last_peered":"2024-04-25T01:01:38.459211+0000","last_clean":"2024-04-25T01:01:38.459211+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:01:38.459211+0000","last_undegraded":"2024-04-25T01:01:38.459211+0000","last_fullsized":"2024-04-25T01:01:38.459211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468702+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:01:39.468702+0000","last_peered":"2024-04-25T01:01:39.468702+0000","last_clean":"2024-04-25T01:01:39.468702+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:01:39.468702+0000","last_undegraded":"2024-04-25T01:01:39.468702+0000","last_fullsized":"2024-04-25T01:01:39.468702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098040+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:01:39.098040+0000","last_peered":"2024-04-25T01:01:39.098040+0000","last_clean":"2024-04-25T01:01:39.098040+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:01:39.098040+0000","last_undegraded":"2024-04-25T01:01:39.098040+0000","last_fullsized":"2024-04-25T01:01:39.098040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417195+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:01:38.417195+0000","last_peered":"2024-04-25T01:01:38.417195+0000","last_clean":"2024-04-25T01:01:38.417195+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:01:38.417195+0000","last_undegraded":"2024-04-25T01:01:38.417195+0000","last_fullsized":"2024-04-25T01:01:38.417195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.576085+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:01:39.576085+0000","last_peered":"2024-04-25T01:01:39.576085+0000","last_clean":"2024-04-25T01:01:39.576085+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:01:39.576085+0000","last_undegraded":"2024-04-25T01:01:39.576085+0000","last_fullsized":"2024-04-25T01:01:39.576085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459745+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:01:38.459745+0000","last_peered":"2024-04-25T01:01:38.459745+0000","last_clean":"2024-04-25T01:01:38.459745+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:01:38.459745+0000","last_undegraded":"2024-04-25T01:01:38.459745+0000","last_fullsized":"2024-04-25T01:01:38.459745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.458917+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:01:38.458917+0000","last_peered":"2024-04-25T01:01:38.458917+0000","last_clean":"2024-04-25T01:01:38.458917+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:01:38.458917+0000","last_undegraded":"2024-04-25T01:01:38.458917+0000","last_fullsized":"2024-04-25T01:01:38.458917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.551583+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:01:38.551583+0000","last_peered":"2024-04-25T01:01:38.551583+0000","last_clean":"2024-04-25T01:01:38.551583+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:01:38.551583+0000","last_undegraded":"2024-04-25T01:01:38.551583+0000","last_fullsized":"2024-04-25T01:01:38.551583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468688+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:01:39.468688+0000","last_peered":"2024-04-25T01:01:39.468688+0000","last_clean":"2024-04-25T01:01:39.468688+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:01:39.468688+0000","last_undegraded":"2024-04-25T01:01:39.468688+0000","last_fullsized":"2024-04-25T01:01:39.468688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415410+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:01:38.415410+0000","last_peered":"2024-04-25T01:01:38.415410+0000","last_clean":"2024-04-25T01:01:38.415410+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:01:38.415410+0000","last_undegraded":"2024-04-25T01:01:38.415410+0000","last_fullsized":"2024-04-25T01:01:38.415410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.416131+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:01:38.416131+0000","last_peered":"2024-04-25T01:01:38.416131+0000","last_clean":"2024-04-25T01:01:38.416131+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:01:38.416131+0000","last_undegraded":"2024-04-25T01:01:38.416131+0000","last_fullsized":"2024-04-25T01:01:38.416131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098109+0000","last_change":"2024-04-25T01:01:33.405682+0000","last_active":"2024-04-25T01:01:39.098109+0000","last_peered":"2024-04-25T01:01:39.098109+0000","last_clean":"2024-04-25T01:01:39.098109+0000","last_became_active":"2024-04-25T01:01:33.405406+0000","last_became_peered":"2024-04-25T01:01:33.405406+0000","last_unstale":"2024-04-25T01:01:39.098109+0000","last_undegraded":"2024-04-25T01:01:39.098109+0000","last_fullsized":"2024-04-25T01:01:39.098109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.098315+0000","last_change":"2024-04-25T01:01:33.414506+0000","last_active":"2024-04-25T01:01:39.098315+0000","last_peered":"2024-04-25T01:01:39.098315+0000","last_clean":"2024-04-25T01:01:39.098315+0000","last_became_active":"2024-04-25T01:01:33.414060+0000","last_became_peered":"2024-04-25T01:01:33.414060+0000","last_unstale":"2024-04-25T01:01:39.098315+0000","last_undegraded":"2024-04-25T01:01:39.098315+0000","last_fullsized":"2024-04-25T01:01:39.098315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417283+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:01:38.417283+0000","last_peered":"2024-04-25T01:01:38.417283+0000","last_clean":"2024-04-25T01:01:38.417283+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:01:38.417283+0000","last_undegraded":"2024-04-25T01:01:38.417283+0000","last_fullsized":"2024-04-25T01:01:38.417283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.414797+0000","last_change":"2024-04-25T01:01:33.408326+0000","last_active":"2024-04-25T01:01:38.414797+0000","last_peered":"2024-04-25T01:01:38.414797+0000","last_clean":"2024-04-25T01:01:38.414797+0000","last_became_active":"2024-04-25T01:01:33.408039+0000","last_became_peered":"2024-04-25T01:01:33.408039+0000","last_unstale":"2024-04-25T01:01:38.414797+0000","last_undegraded":"2024-04-25T01:01:38.414797+0000","last_fullsized":"2024-04-25T01:01:38.414797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415697+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:01:38.415697+0000","last_peered":"2024-04-25T01:01:38.415697+0000","last_clean":"2024-04-25T01:01:38.415697+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:01:38.415697+0000","last_undegraded":"2024-04-25T01:01:38.415697+0000","last_fullsized":"2024-04-25T01:01:38.415697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459077+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:01:38.459077+0000","last_peered":"2024-04-25T01:01:38.459077+0000","last_clean":"2024-04-25T01:01:38.459077+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:01:38.459077+0000","last_undegraded":"2024-04-25T01:01:38.459077+0000","last_fullsized":"2024-04-25T01:01:38.459077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.415902+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:01:38.415902+0000","last_peered":"2024-04-25T01:01:38.415902+0000","last_clean":"2024-04-25T01:01:38.415902+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:01:38.415902+0000","last_undegraded":"2024-04-25T01:01:38.415902+0000","last_fullsized":"2024-04-25T01:01:38.415902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.459266+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:01:38.459266+0000","last_peered":"2024-04-25T01:01:38.459266+0000","last_clean":"2024-04-25T01:01:38.459266+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:01:38.459266+0000","last_undegraded":"2024-04-25T01:01:38.459266+0000","last_fullsized":"2024-04-25T01:01:38.459266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:39.468558+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:01:39.468558+0000","last_peered":"2024-04-25T01:01:39.468558+0000","last_clean":"2024-04-25T01:01:39.468558+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:01:39.468558+0000","last_undegraded":"2024-04-25T01:01:39.468558+0000","last_fullsized":"2024-04-25T01:01:39.468558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.552094+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:01:38.552094+0000","last_peered":"2024-04-25T01:01:38.552094+0000","last_clean":"2024-04-25T01:01:38.552094+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:01:38.552094+0000","last_undegraded":"2024-04-25T01:01:38.552094+0000","last_fullsized":"2024-04-25T01:01:38.552094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:01:38.417169+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:01:38.417169+0000","last_peered":"2024-04-25T01:01:38.417169+0000","last_clean":"2024-04-25T01:01:38.417169+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:01:38.417169+0000","last_undegraded":"2024-04-25T01:01:38.417169+0000","last_fullsized":"2024-04-25T01:01:38.417169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:02:01.109 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T01:02:01.110 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T01:02:01.110 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T01:02:01.110 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T01:02:01.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.237+0000 7f105f66b640 1 Processor -- start 2024-04-25T01:02:01.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.237+0000 7f105f66b640 1 -- start start 2024-04-25T01:02:01.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105f66b640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1058067400 0x7f10580c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105f66b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f105806dff0 con 0x7f1058155e30 2024-04-25T01:02:01.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105f66b640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f105806e2d0 con 0x7f105814bde0 2024-04-25T01:02:01.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105f66b640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f105806edb0 con 0x7f1058067400 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105dc08640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f1058155e30 0x7f10580c0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44820/0 (socket says 172.21.15.129:44820) 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105dc08640 1 -- 172.21.15.129:0/3745203529 learned_addr learned my addr 172.21.15.129:0/3745203529 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105cc06640 1 --2- 172.21.15.129:0/3745203529 >> v2:172.21.15.171:3300/0 conn(0x7f1058067400 0x7f10580c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340154116 0 0) 0x7f105806dff0 con 0x7f1058155e30 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1044003520 con 0x7f1058155e30 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3057634435 0 0) 0x7f1044003520 con 0x7f1058155e30 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 >> v2:172.21.15.171:3300/0 conn(0x7f1058067400 msgr2=0x7f10580c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 --2- 172.21.15.129:0/3745203529 >> v2:172.21.15.171:3300/0 conn(0x7f1058067400 0x7f10580c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 >> v1:172.21.15.171:6789/0 conn(0x7f105814bde0 legacy=0x7f10580c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105806e5b0 con 0x7f1058155e30 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f10500019f0 con 0x7f1058155e30 2024-04-25T01:02:01.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 >> v1:172.21.15.129:6789/0 conn(0x7f1058155e30 legacy=0x7f10580c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f10440039b0 0x7f1044003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1044004800 0x7f1044004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10440051c0 0x7f104400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f105806edb0 con 0x7f1044004800 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f104400c060 con 0x7f10440051c0 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f104400c260 con 0x7f10440039b0 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105cc06640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1044004800 0x7f1044004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.238+0000 7f105d407640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f10440039b0 0x7f1044003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105dc08640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10440051c0 0x7f104400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105cc06640 1 -- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f10440039b0 msgr2=0x7f1044003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105cc06640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f10440039b0 0x7f1044003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105cc06640 1 -- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10440051c0 msgr2=0x7f104400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105cc06640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10440051c0 0x7f104400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105dc08640 1 -- 172.21.15.129:0/3745203529 reap_dead start 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105cc06640 1 -- 172.21.15.129:0/3745203529 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1044003520 con 0x7f1044004800 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f105cc06640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1044004800 0x7f1044004c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1048009ac0 tx=0x7f1044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bf4a832e6ee486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048036070 con 0x7f1044004800 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.239+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048031400 con 0x7f1044004800 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f104ffff640 1 -- 172.21.15.129:0/3745203529 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048009690 con 0x7f1044004800 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 -- 172.21.15.129:0/3745203529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1044004800 msgr2=0x7f1044004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1044004800 0x7f1044004c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1048009ac0 tx=0x7f1044005830 comp rx=0 tx=0).stop 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 -- 172.21.15.129:0/3745203529 shutdown_connections 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 --2- 172.21.15.129:0/3745203529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1044004800 0x7f1044004c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 -- 172.21.15.129:0/3745203529 >> 172.21.15.129:0/3745203529 conn(0x7f10580c71f0 msgr2=0x7f10580c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:01.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 -- 172.21.15.129:0/3745203529 shutdown_connections 2024-04-25T01:02:01.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.240+0000 7f105f66b640 1 -- 172.21.15.129:0/3745203529 wait complete. 2024-04-25T01:02:01.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.241+0000 7f105f66b640 1 Processor -- start 2024-04-25T01:02:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.241+0000 7f105f66b640 1 -- start start 2024-04-25T01:02:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105f66b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1044004800 0x7f10581524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105f66b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 0x7f10581529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105f66b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1058148080 0x7f1058148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105f66b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10580732b0 con 0x7f10581509c0 2024-04-25T01:02:01.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105f66b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f105806e920 con 0x7f1044004800 2024-04-25T01:02:01.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105f66b640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1058073430 con 0x7f1058148080 2024-04-25T01:02:01.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 0x7f10581529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 0x7f10581529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38832/0 (socket says 172.21.15.129:38832) 2024-04-25T01:02:01.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 -- 172.21.15.129:0/4041436744 learned_addr learned my addr 172.21.15.129:0/4041436744 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:01.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105d407640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1044004800 0x7f10581524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105dc08640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1058148080 0x7f1058148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 -- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1058148080 msgr2=0x7f1058148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1058148080 0x7f1058148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 -- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1044004800 msgr2=0x7f10581524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1044004800 0x7f10581524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 -- 172.21.15.129:0/4041436744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058151b40 con 0x7f10581509c0 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105dc08640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1058148080 0x7f1058148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105d407640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1044004800 0x7f10581524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f105cc06640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 0x7f10581529f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1048034040 tx=0x7f1048004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.242+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104803f020 con 0x7f10581509c0 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.243+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048037040 con 0x7f10581509c0 2024-04-25T01:02:01.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.243+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10581494e0 con 0x7f10581509c0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.243+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048036070 con 0x7f10581509c0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.243+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10580aacc0 con 0x7f10581509c0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.244+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105806dff0 con 0x7f10581509c0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.244+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1048042020 con 0x7f10581509c0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.244+0000 7f104dffb640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f103c06be60 0x7f103c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.245+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f10480ab230 con 0x7f10581509c0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.245+0000 7f105d407640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f103c06be60 0x7f103c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.245+0000 7f105d407640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f103c06be60 0x7f103c06e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1054005580 tx=0x7f1054009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.249+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f105806dff0 con 0x7f10581509c0 2024-04-25T01:02:01.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.462+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1058075550 con 0x7f10581509c0 2024-04-25T01:02:01.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.462+0000 7f104dffb640 1 -- 172.21.15.129:0/4041436744 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1048049030 con 0x7f10581509c0 2024-04-25T01:02:01.463 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:01.463 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T01:02:01.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f103c06be60 msgr2=0x7f103c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f103c06be60 0x7f103c06e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1054005580 tx=0x7f1054009040 comp rx=0 tx=0).stop 2024-04-25T01:02:01.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 msgr2=0x7f10581529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 0x7f10581529f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1048034040 tx=0x7f1048004730 comp rx=0 tx=0).stop 2024-04-25T01:02:01.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 shutdown_connections 2024-04-25T01:02:01.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f103c06be60 0x7f103c06e320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1058148080 0x7f1058148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10581509c0 0x7f10581529f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 --2- 172.21.15.129:0/4041436744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1044004800 0x7f10581524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 >> 172.21.15.129:0/4041436744 conn(0x7f10580c71f0 msgr2=0x7f105806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 shutdown_connections 2024-04-25T01:02:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.464+0000 7f105f66b640 1 -- 172.21.15.129:0/4041436744 wait complete. 2024-04-25T01:02:01.476 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T01:02:01.476 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:02:01.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.572+0000 7f80b874f640 1 Processor -- start 2024-04-25T01:02:01.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.572+0000 7f80b874f640 1 -- start start 2024-04-25T01:02:01.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b874f640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b00ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b874f640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f80b01561e0 0x7f80b00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b874f640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b00c9860 con 0x7f80b0152fa0 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b874f640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f80b0156690 con 0x7f80b01561e0 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b874f640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f80b00c5a70 con 0x7f80b00c5520 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b6cec640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f80b01561e0 0x7f80b00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b6cec640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f80b01561e0 0x7f80b00c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38848/0 (socket says 172.21.15.129:38848) 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.573+0000 7f80b6cec640 1 -- 172.21.15.129:0/1870646142 learned_addr learned my addr 172.21.15.129:0/1870646142 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.574+0000 7f80b5cea640 1 --2- 172.21.15.129:0/1870646142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b00ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.574+0000 7f80b6cec640 1 -- 172.21.15.129:0/1870646142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 msgr2=0x7f80b00ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.574+0000 7f80b6cec640 1 --2- 172.21.15.129:0/1870646142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b00ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.574+0000 7f80b6cec640 1 -- 172.21.15.129:0/1870646142 >> v1:172.21.15.171:6789/0 conn(0x7f80b0152fa0 legacy=0x7f80b00c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.574+0000 7f80b6cec640 1 -- 172.21.15.129:0/1870646142 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b00c5d50 con 0x7f80b01561e0 2024-04-25T01:02:01.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.574+0000 7f80b6cec640 1 --2- 172.21.15.129:0/1870646142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b00c8cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c739102dc9cd2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b54e9640 1 -- 172.21.15.129:0/1870646142 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8037070 con 0x7f80b01561e0 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b54e9640 1 -- 172.21.15.129:0/1870646142 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8032440 con 0x7f80b01561e0 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b54e9640 1 -- 172.21.15.129:0/1870646142 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8032810 con 0x7f80b01561e0 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 -- 172.21.15.129:0/1870646142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 msgr2=0x7f80b00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 --2- 172.21.15.129:0/1870646142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b00c8cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a802a130 comp rx=0 tx=0).stop 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 -- 172.21.15.129:0/1870646142 shutdown_connections 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 --2- 172.21.15.129:0/1870646142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b00c8cc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 --2- 172.21.15.129:0/1870646142 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b00ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 -- 172.21.15.129:0/1870646142 >> 172.21.15.129:0/1870646142 conn(0x7f80b006c400 msgr2=0x7f80b006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 -- 172.21.15.129:0/1870646142 shutdown_connections 2024-04-25T01:02:01.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.575+0000 7f80b874f640 1 -- 172.21.15.129:0/1870646142 wait complete. 2024-04-25T01:02:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.576+0000 7f80b874f640 1 Processor -- start 2024-04-25T01:02:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.576+0000 7f80b874f640 1 -- start start 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b874f640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b0157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b874f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80b0152fa0 0x7f80b01eef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b874f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b01ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b874f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80b00c9200 con 0x7f80b01561e0 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b874f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f80b00c6030 con 0x7f80b0152fa0 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b874f640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f80b00c9380 con 0x7f80b00c5520 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b01ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b01ef4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38860/0 (socket says 172.21.15.129:38860) 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 -- 172.21.15.129:0/1083890929 learned_addr learned my addr 172.21.15.129:0/1083890929 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b5cea640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80b0152fa0 0x7f80b01eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b64eb640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b0157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 -- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 msgr2=0x7f80b0157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b0157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 -- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80b0152fa0 msgr2=0x7f80b01eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80b0152fa0 0x7f80b01eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 -- 172.21.15.129:0/1083890929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0158c40 con 0x7f80b01561e0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b5cea640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80b0152fa0 0x7f80b01eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.577+0000 7f80b6cec640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b01ef4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.578+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8037040 con 0x7f80b01561e0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.578+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8042070 con 0x7f80b01561e0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.578+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b00bf870 con 0x7f80b01561e0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.578+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8030900 con 0x7f80b01561e0 2024-04-25T01:02:01.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.578+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b00bec60 con 0x7f80b01561e0 2024-04-25T01:02:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.579+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0075550 con 0x7f80b01561e0 2024-04-25T01:02:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.579+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80a8030aa0 con 0x7f80b01561e0 2024-04-25T01:02:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.579+0000 7f80a6ffd640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f809406be30 0x7f809406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.579+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f80a80aba30 con 0x7f80b01561e0 2024-04-25T01:02:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.580+0000 7f80b64eb640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f809406be30 0x7f809406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.580+0000 7f80b64eb640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f809406be30 0x7f809406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80ac007fd0 tx=0x7f80ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.583+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f80b0158c40 con 0x7f80b01561e0 2024-04-25T01:02:01.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.757+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80b00cb930 con 0x7f80b01561e0 2024-04-25T01:02:01.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.758+0000 7f80a6ffd640 1 -- 172.21.15.129:0/1083890929 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f80a8045360 con 0x7f80b01561e0 2024-04-25T01:02:01.759 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:01.759 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:02:01.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f809406be30 msgr2=0x7f809406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f809406be30 0x7f809406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80ac007fd0 tx=0x7f80ac009040 comp rx=0 tx=0).stop 2024-04-25T01:02:01.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 msgr2=0x7f80b01ef4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b01ef4a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a80046c0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 shutdown_connections 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f809406be30 0x7f809406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b01561e0 0x7f80b01ef4a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80b0152fa0 0x7f80b01eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 --2- 172.21.15.129:0/1083890929 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80b00c5520 0x7f80b0157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 >> 172.21.15.129:0/1083890929 conn(0x7f80b006c400 msgr2=0x7f80b0155860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 shutdown_connections 2024-04-25T01:02:01.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.760+0000 7f80b874f640 1 -- 172.21.15.129:0/1083890929 wait complete. 2024-04-25T01:02:01.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:02:01.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:02:01.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.874+0000 7f9aa025d640 1 Processor -- start 2024-04-25T01:02:01.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.875+0000 7f9aa025d640 1 -- start start 2024-04-25T01:02:01.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.875+0000 7f9aa025d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a98159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.875+0000 7f9aa025d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9a980c9f70 0x7f9a9815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.875+0000 7f9aa025d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a980c94e0 con 0x7f9a980c8cc0 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.875+0000 7f9aa025d640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f9a980c5c10 con 0x7f9a980c9f70 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.875+0000 7f9aa025d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9a980c5650 con 0x7f9a980c90a0 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9a980c9f70 0x7f9a9815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9a980c9f70 0x7f9a9815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38870/0 (socket says 172.21.15.129:38870) 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/2884280455 learned_addr learned my addr 172.21.15.129:0/2884280455 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9d7f8640 1 --2- 172.21.15.129:0/2884280455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a98159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/2884280455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 msgr2=0x7f9a98159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 --2- 172.21.15.129:0/2884280455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a98159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/2884280455 >> v1:172.21.15.171:6789/0 conn(0x7f9a980c8cc0 legacy=0x7f9a98156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:01.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/2884280455 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a980c5930 con 0x7f9a980c9f70 2024-04-25T01:02:01.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.876+0000 7f9a9e7fa640 1 --2- 172.21.15.129:0/2884280455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a9815ff50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009e50 tx=0x7f9a9002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b8ba22aea8d0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9a9cff7640 1 -- 172.21.15.129:0/2884280455 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90036070 con 0x7f9a980c9f70 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9a9cff7640 1 -- 172.21.15.129:0/2884280455 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a90031600 con 0x7f9a980c9f70 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9a9cff7640 1 -- 172.21.15.129:0/2884280455 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a900319d0 con 0x7f9a980c9f70 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 -- 172.21.15.129:0/2884280455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 msgr2=0x7f9a9815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 --2- 172.21.15.129:0/2884280455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a9815ff50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009e50 tx=0x7f9a9002a630 comp rx=0 tx=0).stop 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 -- 172.21.15.129:0/2884280455 shutdown_connections 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 --2- 172.21.15.129:0/2884280455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a9815ff50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 --2- 172.21.15.129:0/2884280455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a98159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 -- 172.21.15.129:0/2884280455 >> 172.21.15.129:0/2884280455 conn(0x7f9a9806c400 msgr2=0x7f9a9806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 -- 172.21.15.129:0/2884280455 shutdown_connections 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.877+0000 7f9aa025d640 1 -- 172.21.15.129:0/2884280455 wait complete. 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.878+0000 7f9aa025d640 1 Processor -- start 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.878+0000 7f9aa025d640 1 -- start start 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9aa025d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a980c8cc0 0x7f9a981ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9aa025d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a9814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9aa025d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a981eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9aa025d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9806d1f0 con 0x7f9a980c9f70 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9aa025d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9a980ca400 con 0x7f9a980c8cc0 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9aa025d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9a980c5f30 con 0x7f9a980c90a0 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a981eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a981eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38874/0 (socket says 172.21.15.129:38874) 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/931681849 learned_addr learned my addr 172.21.15.129:0/931681849 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9dff9640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a980c8cc0 0x7f9a981ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9d7f8640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a9814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 msgr2=0x7f9a9814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a9814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a980c8cc0 msgr2=0x7f9a981ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a980c8cc0 0x7f9a981ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:01.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9e7fa640 1 -- 172.21.15.129:0/931681849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9814c460 con 0x7f9a980c9f70 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.879+0000 7f9a9d7f8640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a9814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.880+0000 7f9a9e7fa640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a981eb2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009f70 tx=0x7f9a90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.880+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90036050 con 0x7f9a980c9f70 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.880+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a90042070 con 0x7f9a980c9f70 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.880+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90009510 con 0x7f9a980c9f70 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.880+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a98159240 con 0x7f9a980c9f70 2024-04-25T01:02:01.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.880+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981ee860 con 0x7f9a980c9f70 2024-04-25T01:02:01.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.881+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a98075550 con 0x7f9a980c9f70 2024-04-25T01:02:01.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.881+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a90007bd0 con 0x7f9a980c9f70 2024-04-25T01:02:01.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.882+0000 7f9a8e7fc640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9a7c06be30 0x7f9a7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:01.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.882+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9a900abff0 con 0x7f9a980c9f70 2024-04-25T01:02:01.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.882+0000 7f9a9dff9640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9a7c06be30 0x7f9a7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:01.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.882+0000 7f9a9dff9640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9a7c06be30 0x7f9a7c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a94006fd0 tx=0x7f9a94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:01.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:01.886+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9a90045360 con 0x7f9a980c9f70 2024-04-25T01:02:02.058 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.058+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a980c9b40 con 0x7f9a980c9f70 2024-04-25T01:02:02.059 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.058+0000 7f9a8e7fc640 1 -- 172.21.15.129:0/931681849 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9a9007a6f0 con 0x7f9a980c9f70 2024-04-25T01:02:02.059 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9a7c06be30 msgr2=0x7f9a7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9a7c06be30 0x7f9a7c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a94006fd0 tx=0x7f9a94008040 comp rx=0 tx=0).stop 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 msgr2=0x7f9a981eb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a981eb2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009f70 tx=0x7f9a90004a40 comp rx=0 tx=0).stop 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 shutdown_connections 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9a7c06be30 0x7f9a7c06e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9a980c90a0 0x7f9a9814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a980c9f70 0x7f9a981eb2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 --2- 172.21.15.129:0/931681849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a980c8cc0 0x7f9a981ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 >> 172.21.15.129:0/931681849 conn(0x7f9a9806c400 msgr2=0x7f9a98151ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 shutdown_connections 2024-04-25T01:02:02.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.060+0000 7f9aa025d640 1 -- 172.21.15.129:0/931681849 wait complete. 2024-04-25T01:02:02.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:02:02.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.172+0000 7f9d62330640 1 Processor -- start 2024-04-25T01:02:02.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d62330640 1 -- start start 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d62330640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9d5c14f6f0 0x7f9d5c0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d62330640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0a6430 con 0x7f9d5c14fad0 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d62330640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0c4930 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d62330640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c0ab270 con 0x7f9d5c14f6f0 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d5b7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f9d5c14fad0 0x7f9d5c152bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44834/0 (socket says 172.21.15.129:44834) 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d5b7fe640 1 -- 172.21.15.129:0/1194659485 learned_addr learned my addr 172.21.15.129:0/1194659485 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.173+0000 7f9d5bfff640 1 --2- 172.21.15.129:0/1194659485 >> v2:172.21.15.171:3300/0 conn(0x7f9d5c14f6f0 0x7f9d5c0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.174+0000 7f9d5affd640 1 -- 172.21.15.129:0/1194659485 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172996741 0 0) 0x7f9d5c0a6430 con 0x7f9d5c14fad0 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.174+0000 7f9d5affd640 1 -- 172.21.15.129:0/1194659485 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d44003520 con 0x7f9d5c14fad0 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.174+0000 7f9d5bfff640 1 -- 172.21.15.129:0/1194659485 >> v1:172.21.15.171:6789/0 conn(0x7f9d5c0c53d0 legacy=0x7f9d5c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.174+0000 7f9d5bfff640 1 -- 172.21.15.129:0/1194659485 >> v1:172.21.15.129:6789/0 conn(0x7f9d5c14fad0 legacy=0x7f9d5c152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.174+0000 7f9d5bfff640 1 -- 172.21.15.129:0/1194659485 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c0c5e20 con 0x7f9d5c14f6f0 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.174+0000 7f9d5bfff640 1 --2- 172.21.15.129:0/1194659485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c0aad30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0c4b00 tx=0x7f9d4c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f5ccca30d8d40c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d5affd640 1 -- 172.21.15.129:0/1194659485 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c037070 con 0x7f9d5c14f6f0 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d5affd640 1 -- 172.21.15.129:0/1194659485 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4c02ae50 con 0x7f9d5c14f6f0 2024-04-25T01:02:02.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d5affd640 1 -- 172.21.15.129:0/1194659485 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c032640 con 0x7f9d5c14f6f0 2024-04-25T01:02:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 -- 172.21.15.129:0/1194659485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 msgr2=0x7f9d5c0aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 --2- 172.21.15.129:0/1194659485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c0aad30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0c4b00 tx=0x7f9d4c02a100 comp rx=0 tx=0).stop 2024-04-25T01:02:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 -- 172.21.15.129:0/1194659485 shutdown_connections 2024-04-25T01:02:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 --2- 172.21.15.129:0/1194659485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c0aad30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.180 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 -- 172.21.15.129:0/1194659485 >> 172.21.15.129:0/1194659485 conn(0x7f9d5c0a86b0 msgr2=0x7f9d5c0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 -- 172.21.15.129:0/1194659485 shutdown_connections 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.175+0000 7f9d62330640 1 -- 172.21.15.129:0/1194659485 wait complete. 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.176+0000 7f9d62330640 1 Processor -- start 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 -- start start 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 0x7f9d5c151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c14fad0 0x7f9d5c146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c4eb0 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c4bd0 con 0x7f9d5c14f6f0 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d62330640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5c06ddd0 con 0x7f9d5c14fad0 2024-04-25T01:02:02.181 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 0x7f9d5c151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 0x7f9d5c151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38878/0 (socket says 172.21.15.129:38878) 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d5bfff640 1 -- 172.21.15.129:0/3313879798 learned_addr learned my addr 172.21.15.129:0/3313879798 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d608cd640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c14fad0 0x7f9d5c146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d5bfff640 1 -- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c14fad0 msgr2=0x7f9d5c146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.177+0000 7f9d5bfff640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c14fad0 0x7f9d5c146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d5bfff640 1 -- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 msgr2=0x7f9d5c145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d5bfff640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d5bfff640 1 -- 172.21.15.129:0/3313879798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c073780 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d608cd640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c14fad0 0x7f9d5c146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d5bfff640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 0x7f9d5c151a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c000c00 tx=0x7f9d4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c037040 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4c004030 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5c0734a0 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c004400 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.178+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5c0713a0 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.179+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c075550 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.179+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d4c003610 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.180+0000 7f9d58ff9640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.180+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9d4c0abea0 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.180+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.180+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c1522b0 tx=0x7f9d50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.184+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d4c07a5a0 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.355+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d5c069070 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.356+0000 7f9d58ff9640 1 -- 172.21.15.129:0/3313879798 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f9d4c079ea0 con 0x7f9d5c0c53d0 2024-04-25T01:02:02.356 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:02:02.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.357+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 msgr2=0x7f9d4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.357+0000 7f9d62330640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c1522b0 tx=0x7f9d50009040 comp rx=0 tx=0).stop 2024-04-25T01:02:02.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 msgr2=0x7f9d5c151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 0x7f9d5c151a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c000c00 tx=0x7f9d4c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 shutdown_connections 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c14fad0 0x7f9d5c146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c14f6f0 0x7f9d5c145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 --2- 172.21.15.129:0/3313879798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c53d0 0x7f9d5c151a80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 >> 172.21.15.129:0/3313879798 conn(0x7f9d5c0a86b0 msgr2=0x7f9d5c152630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 shutdown_connections 2024-04-25T01:02:02.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.358+0000 7f9d62330640 1 -- 172.21.15.129:0/3313879798 wait complete. 2024-04-25T01:02:02.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:02:02.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.468+0000 7f6772d10640 1 Processor -- start 2024-04-25T01:02:02.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6772d10640 1 -- start start 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6772d10640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f676c14f6f0 0x7f676c0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6772d10640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6772d10640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c0a6430 con 0x7f676c14fad0 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6772d10640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f676c0ab270 con 0x7f676c14f6f0 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6772d10640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f676c0c4930 con 0x7f676c0c53d0 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f6770aac640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f676c14f6f0 0x7f676c0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f676bfff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f676c14fad0 0x7f676c152bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44840/0 (socket says 172.21.15.129:44840) 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.469+0000 7f676bfff640 1 -- 172.21.15.129:0/1587317022 learned_addr learned my addr 172.21.15.129:0/1587317022 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f67712ad640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f6770aac640 1 -- 172.21.15.129:0/1587317022 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 msgr2=0x7f676c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f6770aac640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c06d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f6770aac640 1 -- 172.21.15.129:0/1587317022 >> v1:172.21.15.129:6789/0 conn(0x7f676c14fad0 legacy=0x7f676c152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f6770aac640 1 -- 172.21.15.129:0/1587317022 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c0c5e20 con 0x7f676c14f6f0 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f676b7fe640 1 -- 172.21.15.129:0/1587317022 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2521955052 0 0) 0x7f676c0a6430 con 0x7f676c14fad0 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f67712ad640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f6770aac640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14f6f0 0x7f676c0aad30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f67600097a0 tx=0x7f676002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628efd4ca8f50af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f676b7fe640 1 -- 172.21.15.129:0/1587317022 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760037070 con 0x7f676c14f6f0 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f676b7fe640 1 -- 172.21.15.129:0/1587317022 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676002adb0 con 0x7f676c14f6f0 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.470+0000 7f676b7fe640 1 -- 172.21.15.129:0/1587317022 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760032660 con 0x7f676c14f6f0 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 -- 172.21.15.129:0/1587317022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14f6f0 msgr2=0x7f676c0aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14f6f0 0x7f676c0aad30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f67600097a0 tx=0x7f676002a060 comp rx=0 tx=0).stop 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 -- 172.21.15.129:0/1587317022 shutdown_connections 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 --2- 172.21.15.129:0/1587317022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14f6f0 0x7f676c0aad30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 -- 172.21.15.129:0/1587317022 >> 172.21.15.129:0/1587317022 conn(0x7f676c0a86b0 msgr2=0x7f676c0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 -- 172.21.15.129:0/1587317022 shutdown_connections 2024-04-25T01:02:02.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.471+0000 7f6772d10640 1 -- 172.21.15.129:0/1587317022 wait complete. 2024-04-25T01:02:02.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.472+0000 7f6772d10640 1 Processor -- start 2024-04-25T01:02:02.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.472+0000 7f6772d10640 1 -- start start 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6772d10640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c1ea250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6772d10640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f676c14f6f0 0x7f676c1e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6772d10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 0x7f676c1e3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6772d10640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f676c0c4eb0 con 0x7f676c14fad0 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6772d10640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f676c0c4bd0 con 0x7f676c14f6f0 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6772d10640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f676c0c5880 con 0x7f676c0c53d0 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 0x7f676c1e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 0x7f676c1e3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38894/0 (socket says 172.21.15.129:38894) 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 -- 172.21.15.129:0/3968145682 learned_addr learned my addr 172.21.15.129:0/3968145682 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f6770aac640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c1ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 -- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 msgr2=0x7f676c1ea250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c1ea250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f676bfff640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f676c14f6f0 0x7f676c1e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 -- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f676c14f6f0 msgr2=0x7f676c1e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f676c14f6f0 0x7f676c1e75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 -- 172.21.15.129:0/3968145682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c1e6d10 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.473+0000 7f67712ad640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 0x7f676c1e3650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f676400cef0 tx=0x7f676400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.474+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676401a070 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.474+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676400d860 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.474+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676400dc50 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.474+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c1e9770 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.474+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c1e6ff0 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.475+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c075550 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.475+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f676401d070 con 0x7f676c14fad0 2024-04-25T01:02:02.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.475+0000 7f67697fa640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f675006be30 0x7f675006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.476+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f676408b9b0 con 0x7f676c14fad0 2024-04-25T01:02:02.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.476+0000 7f6770aac640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f675006be30 0x7f675006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.476+0000 7f6770aac640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f675006be30 0x7f675006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f676002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.480+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f676405a170 con 0x7f676c14fad0 2024-04-25T01:02:02.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.653+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f676c14d9f0 con 0x7f676c14fad0 2024-04-25T01:02:02.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.653+0000 7f67697fa640 1 -- 172.21.15.129:0/3968145682 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f6764059a70 con 0x7f676c14fad0 2024-04-25T01:02:02.654 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:02:02.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f675006be30 msgr2=0x7f675006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f675006be30 0x7f675006e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f676002e000 comp rx=0 tx=0).stop 2024-04-25T01:02:02.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 msgr2=0x7f676c1e3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 0x7f676c1e3650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f676400cef0 tx=0x7f676400a710 comp rx=0 tx=0).stop 2024-04-25T01:02:02.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 shutdown_connections 2024-04-25T01:02:02.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f675006be30 0x7f675006e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f676c14fad0 0x7f676c1e3650 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f676c14f6f0 0x7f676c1e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 --2- 172.21.15.129:0/3968145682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f676c0c53d0 0x7f676c1ea250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 >> 172.21.15.129:0/3968145682 conn(0x7f676c0a86b0 msgr2=0x7f676c0ab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 shutdown_connections 2024-04-25T01:02:02.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.655+0000 7f6772d10640 1 -- 172.21.15.129:0/3968145682 wait complete. 2024-04-25T01:02:02.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:02:02.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.766+0000 7f1f01a76640 1 Processor -- start 2024-04-25T01:02:02.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.766+0000 7f1f01a76640 1 -- start start 2024-04-25T01:02:02.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1f01a76640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1efc14f6f0 0x7f1efc0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1f01a76640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1efc0a6430 con 0x7f1efc0c53d0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1f01a76640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1efc0c4930 con 0x7f1efc14fad0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1f01a76640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f1efc0ab270 con 0x7f1efc14f6f0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efb7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1efc14f6f0 0x7f1efc0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efbfff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f1efc0c53d0 0x7f1efc06d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44852/0 (socket says 172.21.15.129:44852) 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efb7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f1efc14f6f0 0x7f1efc0aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38898/0 (socket says 172.21.15.129:38898) 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efbfff640 1 -- 172.21.15.129:0/7978890 learned_addr learned my addr 172.21.15.129:0/7978890 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efb7fe640 1 -- 172.21.15.129:0/7978890 >> v1:172.21.15.171:6789/0 conn(0x7f1efc14fad0 legacy=0x7f1efc152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efb7fe640 1 -- 172.21.15.129:0/7978890 >> v1:172.21.15.129:6789/0 conn(0x7f1efc0c53d0 legacy=0x7f1efc06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efa7fc640 1 -- 172.21.15.129:0/7978890 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3269360718 0 0) 0x7f1efc0a6430 con 0x7f1efc0c53d0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.767+0000 7f1efb7fe640 1 -- 172.21.15.129:0/7978890 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc0c5e20 con 0x7f1efc14f6f0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.768+0000 7f1efb7fe640 1 --2- 172.21.15.129:0/7978890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14f6f0 0x7f1efc0aad30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0c4b00 tx=0x7f1eec02c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67e1b63c87adf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.768+0000 7f1efa7fc640 1 -- 172.21.15.129:0/7978890 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec031710 con 0x7f1efc14f6f0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.768+0000 7f1efa7fc640 1 -- 172.21.15.129:0/7978890 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec0318b0 con 0x7f1efc14f6f0 2024-04-25T01:02:02.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.768+0000 7f1efa7fc640 1 -- 172.21.15.129:0/7978890 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec031c80 con 0x7f1efc14f6f0 2024-04-25T01:02:02.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 -- 172.21.15.129:0/7978890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14f6f0 msgr2=0x7f1efc0aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 --2- 172.21.15.129:0/7978890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14f6f0 0x7f1efc0aad30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0c4b00 tx=0x7f1eec02c7b0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 -- 172.21.15.129:0/7978890 shutdown_connections 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 --2- 172.21.15.129:0/7978890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14f6f0 0x7f1efc0aad30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 -- 172.21.15.129:0/7978890 >> 172.21.15.129:0/7978890 conn(0x7f1efc0a86b0 msgr2=0x7f1efc0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 -- 172.21.15.129:0/7978890 shutdown_connections 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.769+0000 7f1f01a76640 1 -- 172.21.15.129:0/7978890 wait complete. 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.770+0000 7f1f01a76640 1 Processor -- start 2024-04-25T01:02:02.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.770+0000 7f1f01a76640 1 -- start start 2024-04-25T01:02:02.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1efc0c53d0 0x7f1efc0c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1efc14f6f0 0x7f1efc1e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 0x7f1efc1459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc0c4eb0 con 0x7f1efc14fad0 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc0c4bd0 con 0x7f1efc0c53d0 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1efc06ddd0 con 0x7f1efc14f6f0 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 0x7f1efc1459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 0x7f1efc1459a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38902/0 (socket says 172.21.15.129:38902) 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 -- 172.21.15.129:0/2658580644 learned_addr learned my addr 172.21.15.129:0/2658580644 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efb7fe640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1efc0c53d0 0x7f1efc0c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efaffd640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1efc14f6f0 0x7f1efc1e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 -- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1efc14f6f0 msgr2=0x7f1efc1e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1efc14f6f0 0x7f1efc1e6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 -- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1efc0c53d0 msgr2=0x7f1efc0c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1efc0c53d0 0x7f1efc0c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 -- 172.21.15.129:0/2658580644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc1e9c40 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efaffd640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1efc14f6f0 0x7f1efc1e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1efbfff640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 0x7f1efc1459a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400c7a0 tx=0x7f1ef400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4018070 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef400a640 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400aab0 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1efc151fd0 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.771+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1efc1ea1c0 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.772+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1efc075550 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.773+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ef401b070 con 0x7f1efc14fad0 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.773+0000 7f1ee3fff640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1ee806be30 0x7f1ee806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.773+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ef408a760 con 0x7f1efc14fad0 2024-04-25T01:02:02.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.773+0000 7f1efb7fe640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1ee806be30 0x7f1ee806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:02.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.774+0000 7f1efb7fe640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1ee806be30 0x7f1ee806e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0a9380 tx=0x7f1eec031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:02.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.777+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ef4058e60 con 0x7f1efc14fad0 2024-04-25T01:02:02.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.949+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1efc1d85f0 con 0x7f1efc14fad0 2024-04-25T01:02:02.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.950+0000 7f1ee3fff640 1 -- 172.21.15.129:0/2658580644 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f1ef4058760 con 0x7f1efc14fad0 2024-04-25T01:02:02.950 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:02:02.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1ee806be30 msgr2=0x7f1ee806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1ee806be30 0x7f1ee806e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0a9380 tx=0x7f1eec031000 comp rx=0 tx=0).stop 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 msgr2=0x7f1efc1459a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 0x7f1efc1459a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400c7a0 tx=0x7f1ef400cc60 comp rx=0 tx=0).stop 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 shutdown_connections 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1ee806be30 0x7f1ee806e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1efc14f6f0 0x7f1efc1e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1efc14fad0 0x7f1efc1459a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 --2- 172.21.15.129:0/2658580644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1efc0c53d0 0x7f1efc0c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.951+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 >> 172.21.15.129:0/2658580644 conn(0x7f1efc0a86b0 msgr2=0x7f1efc1d8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.952+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 shutdown_connections 2024-04-25T01:02:02.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:02.952+0000 7f1f01a76640 1 -- 172.21.15.129:0/2658580644 wait complete. 2024-04-25T01:02:02.963 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:02:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.059+0000 7f0e290f5640 1 Processor -- start 2024-04-25T01:02:03.060 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.059+0000 7f0e290f5640 1 -- start start 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e290f5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e290f5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0e240c4c40 0x7f0e240c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e290f5640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e240c18d0 con 0x7f0e2414f260 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e290f5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2406c480 con 0x7f0e240c4c40 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e290f5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2406c760 con 0x7f0e2406cf60 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2359e640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0e240c4c40 0x7f0e240c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.061 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2259c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e22d9d640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f0e2414f260 0x7f0e2406df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54116/0 (socket says 172.21.15.129:54116) 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2259c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:48032/0 (socket says 172.21.15.129:48032) 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2359e640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0e240c4c40 0x7f0e240c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54962/0 (socket says 172.21.15.129:54962) 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e22d9d640 1 -- 172.21.15.129:0/2361561852 learned_addr learned my addr 172.21.15.129:0/2361561852 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e21d9b640 1 -- 172.21.15.129:0/2361561852 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924891050 0 0) 0x7f0e240c18d0 con 0x7f0e2414f260 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2259c640 1 -- 172.21.15.129:0/2361561852 >> v2:172.21.15.171:3300/0 conn(0x7f0e240c4c40 msgr2=0x7f0e240c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2259c640 1 --2- 172.21.15.129:0/2361561852 >> v2:172.21.15.171:3300/0 conn(0x7f0e240c4c40 0x7f0e240c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2259c640 1 -- 172.21.15.129:0/2361561852 >> v1:172.21.15.171:6789/0 conn(0x7f0e2414f260 legacy=0x7f0e2406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.062 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.060+0000 7f0e2259c640 1 -- 172.21.15.129:0/2361561852 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2406ca40 con 0x7f0e2406cf60 2024-04-25T01:02:03.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.061+0000 7f0e2259c640 1 --2- 172.21.15.129:0/2361561852 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00a940 tx=0x7f0e0c02b4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83fe54774cea75a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.061+0000 7f0e21d9b640 1 -- 172.21.15.129:0/2361561852 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00b040 con 0x7f0e2406cf60 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.061+0000 7f0e21d9b640 1 -- 172.21.15.129:0/2361561852 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c030710 con 0x7f0e2406cf60 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.061+0000 7f0e21d9b640 1 -- 172.21.15.129:0/2361561852 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c030ae0 con 0x7f0e2406cf60 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 -- 172.21.15.129:0/2361561852 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 msgr2=0x7f0e240c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 --2- 172.21.15.129:0/2361561852 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00a940 tx=0x7f0e0c02b4c0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 -- 172.21.15.129:0/2361561852 shutdown_connections 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 --2- 172.21.15.129:0/2361561852 >> v2:172.21.15.171:3300/0 conn(0x7f0e240c4c40 0x7f0e240c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 --2- 172.21.15.129:0/2361561852 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 -- 172.21.15.129:0/2361561852 >> 172.21.15.129:0/2361561852 conn(0x7f0e240a86b0 msgr2=0x7f0e240a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 -- 172.21.15.129:0/2361561852 shutdown_connections 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.062+0000 7f0e290f5640 1 -- 172.21.15.129:0/2361561852 wait complete. 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.063+0000 7f0e290f5640 1 Processor -- start 2024-04-25T01:02:03.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.063+0000 7f0e290f5640 1 -- start start 2024-04-25T01:02:03.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e290f5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e290f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 0x7f0e24072a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e290f5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e2414f260 0x7f0e240ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e290f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2406e7e0 con 0x7f0e240c4c40 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e290f5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2406e660 con 0x7f0e2414f260 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e290f5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240c1300 con 0x7f0e2406cf60 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 0x7f0e24072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 0x7f0e24072a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38914/0 (socket says 172.21.15.129:38914) 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 -- 172.21.15.129:0/1567404390 learned_addr learned my addr 172.21.15.129:0/1567404390 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e22d9d640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2359e640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e2414f260 0x7f0e240ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 -- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 msgr2=0x7f0e240c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 -- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e2414f260 msgr2=0x7f0e240ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e2414f260 0x7f0e240ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.064+0000 7f0e2259c640 1 -- 172.21.15.129:0/1567404390 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e240c56b0 con 0x7f0e240c4c40 2024-04-25T01:02:03.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.065+0000 7f0e2259c640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 0x7f0e24072a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00a940 tx=0x7f0e0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.065+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c040070 con 0x7f0e240c4c40 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.065+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c030e10 con 0x7f0e240c4c40 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.065+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2414f800 con 0x7f0e240c4c40 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.065+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c006e10 con 0x7f0e240c4c40 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.065+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e240c5950 con 0x7f0e240c4c40 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.066+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e24075550 con 0x7f0e240c4c40 2024-04-25T01:02:03.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.066+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e0c043070 con 0x7f0e240c4c40 2024-04-25T01:02:03.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.066+0000 7f0e1b7fe640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e0406be30 0x7f0e0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.067+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e0c0abd10 con 0x7f0e240c4c40 2024-04-25T01:02:03.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.067+0000 7f0e22d9d640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e0406be30 0x7f0e0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.067+0000 7f0e22d9d640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e0406be30 0x7f0e0406e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0012e0 tx=0x7f0e1c001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.071+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e0c079dc0 con 0x7f0e240c4c40 2024-04-25T01:02:03.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.281+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e240c44a0 con 0x7f0e240c4c40 2024-04-25T01:02:03.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.281+0000 7f0e1b7fe640 1 -- 172.21.15.129:0/1567404390 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f0e0c07e0c0 con 0x7f0e240c4c40 2024-04-25T01:02:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:03.282 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:54.802314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:02:03.282 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-25T01:02:03.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.283+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e0406be30 msgr2=0x7f0e0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.283+0000 7f0e290f5640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e0406be30 0x7f0e0406e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0012e0 tx=0x7f0e1c001630 comp rx=0 tx=0).stop 2024-04-25T01:02:03.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.283+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 msgr2=0x7f0e24072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.283+0000 7f0e290f5640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 0x7f0e24072a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00a940 tx=0x7f0e0c004730 comp rx=0 tx=0).stop 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 shutdown_connections 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0e0406be30 0x7f0e0406e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e2414f260 0x7f0e240ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e240c4c40 0x7f0e24072a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 --2- 172.21.15.129:0/1567404390 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e2406cf60 0x7f0e240c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 >> 172.21.15.129:0/1567404390 conn(0x7f0e240a86b0 msgr2=0x7f0e240a97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 shutdown_connections 2024-04-25T01:02:03.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.284+0000 7f0e290f5640 1 -- 172.21.15.129:0/1567404390 wait complete. 2024-04-25T01:02:03.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:02:03.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.392+0000 7f0a421ce640 1 Processor -- start 2024-04-25T01:02:03.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.392+0000 7f0a421ce640 1 -- start start 2024-04-25T01:02:03.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a421ce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c14f6f0 0x7f0a3c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a421ce640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a3c0abe50 0x7f0a3c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a421ce640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a3c0c4f10 con 0x7f0a3c0ac540 2024-04-25T01:02:03.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a421ce640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0a3c0ac2a0 con 0x7f0a3c0abe50 2024-04-25T01:02:03.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a421ce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3c0c5180 con 0x7f0a3c14f6f0 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a3c0abe50 0x7f0a3c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0a3c0abe50 0x7f0a3c06d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38928/0 (socket says 172.21.15.129:38928) 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 -- 172.21.15.129:0/435159405 learned_addr learned my addr 172.21.15.129:0/435159405 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3b7fe640 1 --2- 172.21.15.129:0/435159405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c14f6f0 0x7f0a3c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 -- 172.21.15.129:0/435159405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c14f6f0 msgr2=0x7f0a3c14fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/435159405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c14f6f0 0x7f0a3c14fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 -- 172.21.15.129:0/435159405 >> v1:172.21.15.171:6789/0 conn(0x7f0a3c0ac540 legacy=0x7f0a3c0c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.393+0000 7f0a3bfff640 1 -- 172.21.15.129:0/435159405 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c155720 con 0x7f0a3c0abe50 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/435159405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c06d6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009e50 tx=0x7f0a3402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72cefe7f1c4e0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a3a7fc640 1 -- 172.21.15.129:0/435159405 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34036070 con 0x7f0a3c0abe50 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a3a7fc640 1 -- 172.21.15.129:0/435159405 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a34031600 con 0x7f0a3c0abe50 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a3a7fc640 1 -- 172.21.15.129:0/435159405 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340319d0 con 0x7f0a3c0abe50 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a421ce640 1 -- 172.21.15.129:0/435159405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 msgr2=0x7f0a3c06d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a421ce640 1 --2- 172.21.15.129:0/435159405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c06d6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009e50 tx=0x7f0a3402a680 comp rx=0 tx=0).stop 2024-04-25T01:02:03.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a421ce640 1 -- 172.21.15.129:0/435159405 shutdown_connections 2024-04-25T01:02:03.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a421ce640 1 --2- 172.21.15.129:0/435159405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c06d6c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.394+0000 7f0a421ce640 1 --2- 172.21.15.129:0/435159405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c14f6f0 0x7f0a3c14fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.395+0000 7f0a421ce640 1 -- 172.21.15.129:0/435159405 >> 172.21.15.129:0/435159405 conn(0x7f0a3c0a86b0 msgr2=0x7f0a3c0a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.395+0000 7f0a421ce640 1 -- 172.21.15.129:0/435159405 shutdown_connections 2024-04-25T01:02:03.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.395+0000 7f0a421ce640 1 -- 172.21.15.129:0/435159405 wait complete. 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 Processor -- start 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 -- start start 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c0ac540 0x7f0a3c1e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a3c14f6f0 0x7f0a3c1e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c155cc0 con 0x7f0a3c0abe50 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1559e0 con 0x7f0a3c14f6f0 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a421ce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3c155fa0 con 0x7f0a3c0ac540 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a3b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.396+0000 7f0a3b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c14be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38944/0 (socket says 172.21.15.129:38944) 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 -- 172.21.15.129:0/3576888788 learned_addr learned my addr 172.21.15.129:0/3576888788 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3bfff640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a3c14f6f0 0x7f0a3c1e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3affd640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c0ac540 0x7f0a3c1e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 -- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c0ac540 msgr2=0x7f0a3c1e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c0ac540 0x7f0a3c1e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 -- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a3c14f6f0 msgr2=0x7f0a3c1e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a3c14f6f0 0x7f0a3c1e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 -- 172.21.15.129:0/3576888788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1e97c0 con 0x7f0a3c0abe50 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a3b7fe640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c14be90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d800 tx=0x7f0a2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00b6c0 con 0x7f0a3c0abe50 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c019070 con 0x7f0a3c0abe50 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1e8840 con 0x7f0a3c0abe50 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00ba80 con 0x7f0a3c0abe50 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.397+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c0c1b40 con 0x7f0a3c0abe50 2024-04-25T01:02:03.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.398+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c075550 con 0x7f0a3c0abe50 2024-04-25T01:02:03.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.399+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a2c00bc20 con 0x7f0a3c0abe50 2024-04-25T01:02:03.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.399+0000 7f0a23fff640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2806be30 0x7f0a2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.399+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0a2c08b480 con 0x7f0a3c0abe50 2024-04-25T01:02:03.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.399+0000 7f0a3affd640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2806be30 0x7f0a2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.400+0000 7f0a3affd640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2806be30 0x7f0a2806e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c14f2f0 tx=0x7f0a30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.403+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a3c1e97c0 con 0x7f0a3c0abe50 2024-04-25T01:02:03.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.580+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a3c0c4390 con 0x7f0a3c0abe50 2024-04-25T01:02:03.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.581+0000 7f0a23fff640 1 -- 172.21.15.129:0/3576888788 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f0a2c059530 con 0x7f0a3c0abe50 2024-04-25T01:02:03.582 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:03.582 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2806be30 msgr2=0x7f0a2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2806be30 0x7f0a2806e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c14f2f0 tx=0x7f0a30009040 comp rx=0 tx=0).stop 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 msgr2=0x7f0a3c14be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c14be90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d800 tx=0x7f0a2c00dcc0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 shutdown_connections 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0a2806be30 0x7f0a2806e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a3c0ac540 0x7f0a3c1e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a3c14f6f0 0x7f0a3c1e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 --2- 172.21.15.129:0/3576888788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a3c0abe50 0x7f0a3c14be90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 >> 172.21.15.129:0/3576888788 conn(0x7f0a3c0a86b0 msgr2=0x7f0a3c152220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 shutdown_connections 2024-04-25T01:02:03.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.583+0000 7f0a421ce640 1 -- 172.21.15.129:0/3576888788 wait complete. 2024-04-25T01:02:03.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:02:03.595 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:02:03.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.694+0000 7f5e8c0b3640 1 Processor -- start 2024-04-25T01:02:03.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8c0b3640 1 -- start start 2024-04-25T01:02:03.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8c0b3640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5e8414f6f0 0x7f5e840aad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8c0b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 0x7f5e8406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8c0b3640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e840a6430 con 0x7f5e8414fad0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8c0b3640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f5e840ab270 con 0x7f5e8414f6f0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8c0b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840c4930 con 0x7f5e840c53d0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8964e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f5e8414fad0 0x7f5e84152bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44858/0 (socket says 172.21.15.129:44858) 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.695+0000 7f5e8964e640 1 -- 172.21.15.129:0/3695349808 learned_addr learned my addr 172.21.15.129:0/3695349808 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/3695349808 >> v2:172.21.15.171:3300/0 conn(0x7f5e8414f6f0 0x7f5e840aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8a650640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 0x7f5e8406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387549778 0 0) 0x7f5e840a6430 con 0x7f5e8414fad0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e70003520 con 0x7f5e8414fad0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1031216854 0 0) 0x7f5e70003520 con 0x7f5e8414fad0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 msgr2=0x7f5e8406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 0x7f5e8406d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 >> v2:172.21.15.171:3300/0 conn(0x7f5e8414f6f0 msgr2=0x7f5e840aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 --2- 172.21.15.129:0/3695349808 >> v2:172.21.15.171:3300/0 conn(0x7f5e8414f6f0 0x7f5e840aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e840c5e20 con 0x7f5e8414fad0 2024-04-25T01:02:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f5e80001670 con 0x7f5e8414fad0 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 >> v1:172.21.15.129:6789/0 conn(0x7f5e8414fad0 legacy=0x7f5e84152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e700039b0 0x7f5e70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e70004800 0x7f5e70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 0x7f5e700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840c4930 con 0x7f5e70004800 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7000b9c0 con 0x7f5e700039b0 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7000bbc0 con 0x7f5e840c53d0 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/3695349808 >> v2:172.21.15.171:3300/0 conn(0x7f5e8414f6f0 0x7f5e840aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8964e640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e70004800 0x7f5e70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8a650640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 0x7f5e700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e700039b0 0x7f5e70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8964e640 1 -- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 msgr2=0x7f5e700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8964e640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840c53d0 0x7f5e700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8964e640 1 -- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e700039b0 msgr2=0x7f5e70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8964e640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e700039b0 0x7f5e70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e8964e640 1 -- 172.21.15.129:0/3695349808 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e70003520 con 0x7f5e70004800 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.696+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e700039b0 0x7f5e70003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.697+0000 7f5e8964e640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e70004800 0x7f5e70004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5e80007fd0 tx=0x7f5e70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75ddb87e87bccc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.697+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80007d50 con 0x7f5e70004800 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.697+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e80005af0 con 0x7f5e70004800 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.697+0000 7f5e88e4d640 1 -- 172.21.15.129:0/3695349808 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80007070 con 0x7f5e70004800 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/3695349808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e70004800 msgr2=0x7f5e70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/3695349808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e70004800 0x7f5e70004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5e80007fd0 tx=0x7f5e70005830 comp rx=0 tx=0).stop 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8a650640 1 -- 172.21.15.129:0/3695349808 reap_dead start 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/3695349808 shutdown_connections 2024-04-25T01:02:03.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/3695349808 >> 172.21.15.129:0/3695349808 conn(0x7f5e840a86b0 msgr2=0x7f5e840a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/3695349808 shutdown_connections 2024-04-25T01:02:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.698+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/3695349808 wait complete. 2024-04-25T01:02:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 Processor -- start 2024-04-25T01:02:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 -- start start 2024-04-25T01:02:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e70004800 0x7f5e841ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840bffa0 0x7f5e84151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 0x7f5e84145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840c4eb0 con 0x7f5e841520c0 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840c4bd0 con 0x7f5e70004800 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.699+0000 7f5e8c0b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840c5880 con 0x7f5e840bffa0 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 0x7f5e84145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 0x7f5e84145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38968/0 (socket says 172.21.15.129:38968) 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 -- 172.21.15.129:0/2403631343 learned_addr learned my addr 172.21.15.129:0/2403631343 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e70004800 0x7f5e841ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8964e640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840bffa0 0x7f5e84151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 -- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840bffa0 msgr2=0x7f5e84151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840bffa0 0x7f5e84151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 -- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e70004800 msgr2=0x7f5e841ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e70004800 0x7f5e841ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 -- 172.21.15.129:0/2403631343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e841de350 con 0x7f5e841520c0 2024-04-25T01:02:03.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8a650640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 0x7f5e84145b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e840a9770 tx=0x7f5e7c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00fde0 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c015430 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e841de630 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c015820 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.700+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e840ab2d0 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.701+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e84075550 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.702+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e7c0159c0 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.702+0000 7f5e7a7fc640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5e6806be30 0x7f5e6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.702+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e7c08bf10 con 0x7f5e841520c0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.702+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5e6806be30 0x7f5e6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.703+0000 7f5e89e4f640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5e6806be30 0x7f5e6806e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009ba0 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.707+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e7c019030 con 0x7f5e841520c0 2024-04-25T01:02:03.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.877+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e840ab4a0 con 0x7f5e841520c0 2024-04-25T01:02:03.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.878+0000 7f5e7a7fc640 1 -- 172.21.15.129:0/2403631343 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f5e7c010040 con 0x7f5e841520c0 2024-04-25T01:02:03.878 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:02:03.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.879+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5e6806be30 msgr2=0x7f5e6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5e6806be30 0x7f5e6806e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009ba0 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2024-04-25T01:02:03.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 msgr2=0x7f5e84145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 0x7f5e84145b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5e840a9770 tx=0x7f5e7c00f580 comp rx=0 tx=0).stop 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 shutdown_connections 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5e6806be30 0x7f5e6806e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5e840bffa0 0x7f5e84151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e841520c0 0x7f5e84145b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 --2- 172.21.15.129:0/2403631343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e70004800 0x7f5e841ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 >> 172.21.15.129:0/2403631343 conn(0x7f5e840a86b0 msgr2=0x7f5e840c1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 shutdown_connections 2024-04-25T01:02:03.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.880+0000 7f5e8c0b3640 1 -- 172.21.15.129:0/2403631343 wait complete. 2024-04-25T01:02:03.891 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:02:03.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.991+0000 7faeb0d1b640 1 Processor -- start 2024-04-25T01:02:03.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.991+0000 7faeb0d1b640 1 -- start start 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.991+0000 7faeb0d1b640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7faeac067300 0x7faeac0c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.991+0000 7faeb0d1b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac0733a0 con 0x7faeac14f260 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.991+0000 7faeb0d1b640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac073680 con 0x7faeac155d70 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.991+0000 7faeb0d1b640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7faeac158c50 con 0x7faeac067300 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faeaa59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7faeac14f260 0x7faeac1590e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44870/0 (socket says 172.21.15.129:44870) 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faeaa59c640 1 -- 172.21.15.129:0/3175377399 learned_addr learned my addr 172.21.15.129:0/3175377399 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea9d9b640 1 --2- 172.21.15.129:0/3175377399 >> v2:172.21.15.171:3300/0 conn(0x7faeac067300 0x7faeac0c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea959a640 1 -- 172.21.15.129:0/3175377399 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113452921 0 0) 0x7faeac0733a0 con 0x7faeac14f260 2024-04-25T01:02:03.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea959a640 1 -- 172.21.15.129:0/3175377399 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae94003520 con 0x7faeac14f260 2024-04-25T01:02:03.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea9d9b640 1 -- 172.21.15.129:0/3175377399 >> v1:172.21.15.171:6789/0 conn(0x7faeac155d70 legacy=0x7faeac0c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea959a640 1 -- 172.21.15.129:0/3175377399 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 969388670 0 0) 0x7fae94003520 con 0x7faeac14f260 2024-04-25T01:02:03.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea9d9b640 1 -- 172.21.15.129:0/3175377399 >> v1:172.21.15.129:6789/0 conn(0x7faeac14f260 legacy=0x7faeac1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.992+0000 7faea9d9b640 1 -- 172.21.15.129:0/3175377399 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac1588e0 con 0x7faeac067300 2024-04-25T01:02:03.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.993+0000 7faea9d9b640 1 --2- 172.21.15.129:0/3175377399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac0c0620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faea00097d0 tx=0x7faea002a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e964ed8c207739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.993+0000 7faea959a640 1 -- 172.21.15.129:0/3175377399 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0037070 con 0x7faeac067300 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.993+0000 7faea959a640 1 -- 172.21.15.129:0/3175377399 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea002abf0 con 0x7faeac067300 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.993+0000 7faea959a640 1 -- 172.21.15.129:0/3175377399 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0019f0 con 0x7faeac067300 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.993+0000 7faeb0d1b640 1 -- 172.21.15.129:0/3175377399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 msgr2=0x7faeac0c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.993+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/3175377399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac0c0620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faea00097d0 tx=0x7faea002a160 comp rx=0 tx=0).stop 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.994+0000 7faeb0d1b640 1 -- 172.21.15.129:0/3175377399 shutdown_connections 2024-04-25T01:02:03.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.994+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/3175377399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac0c0620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.994+0000 7faeb0d1b640 1 -- 172.21.15.129:0/3175377399 >> 172.21.15.129:0/3175377399 conn(0x7faeac0c71f0 msgr2=0x7faeac0c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:03.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.994+0000 7faeb0d1b640 1 -- 172.21.15.129:0/3175377399 shutdown_connections 2024-04-25T01:02:03.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.994+0000 7faeb0d1b640 1 -- 172.21.15.129:0/3175377399 wait complete. 2024-04-25T01:02:03.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 Processor -- start 2024-04-25T01:02:03.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 -- start start 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faeac14f260 0x7faeac1eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 0x7faeac1ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faeac0c2300 con 0x7faeac155d70 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faeac06b400 con 0x7faeac067300 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.995+0000 7faeb0d1b640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7faeac0c2480 con 0x7faeac14f260 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 0x7faeac1ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 0x7faeac1ebb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38976/0 (socket says 172.21.15.129:38976) 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 -- 172.21.15.129:0/1325811321 learned_addr learned my addr 172.21.15.129:0/1325811321 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaa59c640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faea9d9b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faeac14f260 0x7faeac1eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 -- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faeac14f260 msgr2=0x7faeac1eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faeac14f260 0x7faeac1eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 -- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 msgr2=0x7faeac158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 -- 172.21.15.129:0/1325811321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac1eaf50 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaa59c640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeaad9d640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 0x7faeac1ebb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faea400c3c0 tx=0x7faea400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4017070 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea40052d0 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4003810 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac1488b0 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.996+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac148e70 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.997+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac075550 con 0x7faeac155d70 2024-04-25T01:02:03.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.998+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faea401a070 con 0x7faeac155d70 2024-04-25T01:02:04.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.998+0000 7fae9affd640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fae8c06be30 0x7fae8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.998+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faea408a3f0 con 0x7faeac155d70 2024-04-25T01:02:04.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.998+0000 7faeaa59c640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fae8c06be30 0x7fae8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:03.999+0000 7faeaa59c640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fae8c06be30 0x7fae8c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005c00 tx=0x7fae9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.002+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faea4058af0 con 0x7faeac155d70 2024-04-25T01:02:04.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.175+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faeac14cfd0 con 0x7faeac155d70 2024-04-25T01:02:04.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.176+0000 7fae9affd640 1 -- 172.21.15.129:0/1325811321 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7faea40583f0 con 0x7faeac155d70 2024-04-25T01:02:04.176 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fae8c06be30 msgr2=0x7fae8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fae8c06be30 0x7fae8c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005c00 tx=0x7fae9c009040 comp rx=0 tx=0).stop 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 msgr2=0x7faeac1ebb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 0x7faeac1ebb50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faea400c3c0 tx=0x7faea400c880 comp rx=0 tx=0).stop 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 shutdown_connections 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fae8c06be30 0x7fae8c06e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faeac14f260 0x7faeac1eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faeac155d70 0x7faeac1ebb50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 --2- 172.21.15.129:0/1325811321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faeac067300 0x7faeac158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 >> 172.21.15.129:0/1325811321 conn(0x7faeac0c71f0 msgr2=0x7faeac06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 shutdown_connections 2024-04-25T01:02:04.179 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.178+0000 7faeb0d1b640 1 -- 172.21.15.129:0/1325811321 wait complete. 2024-04-25T01:02:04.189 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:02:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.288+0000 7fa3590d6640 1 Processor -- start 2024-04-25T01:02:04.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.288+0000 7fa3590d6640 1 -- start start 2024-04-25T01:02:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa3590d6640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa35415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa3590d6640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540a61e0 con 0x7fa354153e90 2024-04-25T01:02:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa3590d6640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3541533b0 con 0x7fa354150750 2024-04-25T01:02:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa3590d6640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa354153690 con 0x7fa3540a5ce0 2024-04-25T01:02:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa35259c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fa354153e90 0x7fa354156790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44876/0 (socket says 172.21.15.129:44876) 2024-04-25T01:02:04.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa35259c640 1 -- 172.21.15.129:0/276688291 learned_addr learned my addr 172.21.15.129:0/276688291 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa35359e640 1 --2- 172.21.15.129:0/276688291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa35415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101511708 0 0) 0x7fa3540a61e0 con 0x7fa354153e90 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa33c003520 con 0x7fa354153e90 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093566355 0 0) 0x7fa3541533b0 con 0x7fa354150750 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.289+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3540a61e0 con 0x7fa354150750 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa35359e640 1 -- 172.21.15.129:0/276688291 >> v1:172.21.15.171:6789/0 conn(0x7fa354150750 legacy=0x7fa3540abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa35359e640 1 -- 172.21.15.129:0/276688291 >> v1:172.21.15.129:6789/0 conn(0x7fa354153e90 legacy=0x7fa354156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa35359e640 1 -- 172.21.15.129:0/276688291 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354153970 con 0x7fa3540a5ce0 2024-04-25T01:02:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa35359e640 1 --2- 172.21.15.129:0/276688291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa35415eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009dc0 tx=0x7fa34c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb0e7f567d701b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c007740 con 0x7fa3540a5ce0 2024-04-25T01:02:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c0078e0 con 0x7fa3540a5ce0 2024-04-25T01:02:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.290+0000 7fa351d9b640 1 -- 172.21.15.129:0/276688291 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c007cb0 con 0x7fa3540a5ce0 2024-04-25T01:02:04.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 -- 172.21.15.129:0/276688291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 msgr2=0x7fa35415eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 --2- 172.21.15.129:0/276688291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa35415eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009dc0 tx=0x7fa34c02a6d0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 -- 172.21.15.129:0/276688291 shutdown_connections 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 --2- 172.21.15.129:0/276688291 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa35415eb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 -- 172.21.15.129:0/276688291 >> 172.21.15.129:0/276688291 conn(0x7fa3540a86b0 msgr2=0x7fa3540a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 -- 172.21.15.129:0/276688291 shutdown_connections 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.291+0000 7fa3590d6640 1 -- 172.21.15.129:0/276688291 wait complete. 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.292+0000 7fa3590d6640 1 Processor -- start 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.292+0000 7fa3590d6640 1 -- start start 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa3590d6640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa354070a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa3590d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 0x7fa3541e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa3590d6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa354153e90 0x7fa354157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa3590d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540ac550 con 0x7fa354150750 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa3590d6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540ac3d0 con 0x7fa354153e90 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa3590d6640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa35406c270 con 0x7fa3540a5ce0 2024-04-25T01:02:04.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa35259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 0x7fa3541e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa35259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 0x7fa3541e94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38982/0 (socket says 172.21.15.129:38982) 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa35259c640 1 -- 172.21.15.129:0/3152935951 learned_addr learned my addr 172.21.15.129:0/3152935951 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa35359e640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa354153e90 0x7fa354157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.293+0000 7fa352d9d640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa354070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa35259c640 1 -- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 msgr2=0x7fa354070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa35259c640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa354070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa35259c640 1 -- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa354153e90 msgr2=0x7fa354157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa35259c640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa354153e90 0x7fa354157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa35259c640 1 -- 172.21.15.129:0/3152935951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3541e9a20 con 0x7fa354150750 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa35259c640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 0x7fa3541e94e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c3e0 tx=0x7fa34800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348017070 con 0x7fa354150750 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348005ce0 con 0x7fa354150750 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3541575d0 con 0x7fa354150750 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480072d0 con 0x7fa354150750 2024-04-25T01:02:04.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.294+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3540c60f0 con 0x7fa354150750 2024-04-25T01:02:04.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.295+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354075550 con 0x7fa354150750 2024-04-25T01:02:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.295+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa34801a070 con 0x7fa354150750 2024-04-25T01:02:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.296+0000 7fa33b7fe640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa34006be60 0x7fa34006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.296+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa34808a3a0 con 0x7fa354150750 2024-04-25T01:02:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.296+0000 7fa352d9d640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa34006be60 0x7fa34006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.296+0000 7fa352d9d640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa34006be60 0x7fa34006e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.300+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3541e9a20 con 0x7fa354150750 2024-04-25T01:02:04.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.474+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3540c66e0 con 0x7fa354150750 2024-04-25T01:02:04.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.474+0000 7fa33b7fe640 1 -- 172.21.15.129:0/3152935951 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fa348058450 con 0x7fa354150750 2024-04-25T01:02:04.475 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:02:04.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa34006be60 msgr2=0x7fa34006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa34006be60 0x7fa34006e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa344007fd0 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-25T01:02:04.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 msgr2=0x7fa3541e94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 0x7fa3541e94e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c3e0 tx=0x7fa34800c8a0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 shutdown_connections 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa34006be60 0x7fa34006e320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa354153e90 0x7fa354157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa354150750 0x7fa3541e94e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 --2- 172.21.15.129:0/3152935951 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa3540a5ce0 0x7fa354070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.476+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 >> 172.21.15.129:0/3152935951 conn(0x7fa3540a86b0 msgr2=0x7fa3540a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.477+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 shutdown_connections 2024-04-25T01:02:04.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.477+0000 7fa3590d6640 1 -- 172.21.15.129:0/3152935951 wait complete. 2024-04-25T01:02:04.488 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:02:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.587+0000 7f49fda53640 1 Processor -- start 2024-04-25T01:02:04.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.587+0000 7f49fda53640 1 -- start start 2024-04-25T01:02:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.588+0000 7f49fda53640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 0x7f49f80aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.588+0000 7f49fda53640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f49f8070fd0 0x7f49f80c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.588+0000 7f49fda53640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f80c4960 con 0x7f49f80c4430 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.588+0000 7f49fda53640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f49f8070290 con 0x7f49f8070fd0 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.588+0000 7f49fda53640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f49f80704f0 con 0x7f49f8156460 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.588+0000 7f49f6ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 0x7f49f80aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f67fc640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f49f8070fd0 0x7f49f80c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f6ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 0x7f49f80aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:48106/0 (socket says 172.21.15.129:48106) 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f6ffd640 1 -- 172.21.15.129:0/326755415 learned_addr learned my addr 172.21.15.129:0/326755415 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f77fe640 1 --1- 172.21.15.129:0/326755415 >> v1:172.21.15.171:6789/0 conn(0x7f49f80c4430 0x7f49f80c33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:54136/0 (socket says 172.21.15.129:54136) 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f6ffd640 1 -- 172.21.15.129:0/326755415 >> v2:172.21.15.171:3300/0 conn(0x7f49f8070fd0 msgr2=0x7f49f80c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f6ffd640 1 --2- 172.21.15.129:0/326755415 >> v2:172.21.15.171:3300/0 conn(0x7f49f8070fd0 0x7f49f80c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f6ffd640 1 -- 172.21.15.129:0/326755415 >> v1:172.21.15.171:6789/0 conn(0x7f49f80c4430 legacy=0x7f49f80c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.589+0000 7f49f6ffd640 1 -- 172.21.15.129:0/326755415 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f80707d0 con 0x7f49f8156460 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49f6ffd640 1 --2- 172.21.15.129:0/326755415 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 0x7f49f80aa6d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009e50 tx=0x7f49e802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78bff0a71fd861f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49f5ffb640 1 -- 172.21.15.129:0/326755415 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8036070 con 0x7f49f8156460 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49f5ffb640 1 -- 172.21.15.129:0/326755415 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8031600 con 0x7f49f8156460 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49f5ffb640 1 -- 172.21.15.129:0/326755415 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80319d0 con 0x7f49f8156460 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 -- 172.21.15.129:0/326755415 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 msgr2=0x7f49f80aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 --2- 172.21.15.129:0/326755415 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 0x7f49f80aa6d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009e50 tx=0x7f49e802a680 comp rx=0 tx=0).stop 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 -- 172.21.15.129:0/326755415 shutdown_connections 2024-04-25T01:02:04.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 --2- 172.21.15.129:0/326755415 >> v2:172.21.15.171:3300/0 conn(0x7f49f8070fd0 0x7f49f80c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 --2- 172.21.15.129:0/326755415 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f8156460 0x7f49f80aa6d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 -- 172.21.15.129:0/326755415 >> 172.21.15.129:0/326755415 conn(0x7f49f8066ea0 msgr2=0x7f49f80672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 -- 172.21.15.129:0/326755415 shutdown_connections 2024-04-25T01:02:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.590+0000 7f49fda53640 1 -- 172.21.15.129:0/326755415 wait complete. 2024-04-25T01:02:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.591+0000 7f49fda53640 1 Processor -- start 2024-04-25T01:02:04.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 -- start start 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 0x7f49f814f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f80c4430 0x7f49f80acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49f8156460 0x7f49f81eb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80c2160 con 0x7f49f8070fd0 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80c1fe0 con 0x7f49f8156460 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49fda53640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f49f80c2b50 con 0x7f49f80c4430 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49f6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 0x7f49f814f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49f6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 0x7f49f814f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38992/0 (socket says 172.21.15.129:38992) 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.592+0000 7f49f6ffd640 1 -- 172.21.15.129:0/2306383633 learned_addr learned my addr 172.21.15.129:0/2306383633 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f77fe640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49f8156460 0x7f49f81eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f67fc640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f80c4430 0x7f49f80acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f6ffd640 1 -- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f80c4430 msgr2=0x7f49f80acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f6ffd640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f80c4430 0x7f49f80acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f6ffd640 1 -- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49f8156460 msgr2=0x7f49f81eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f6ffd640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49f8156460 0x7f49f81eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f6ffd640 1 -- 172.21.15.129:0/2306383633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81ebf10 con 0x7f49f8070fd0 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f67fc640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f80c4430 0x7f49f80acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f77fe640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49f8156460 0x7f49f81eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49f6ffd640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 0x7f49f814f9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009f70 tx=0x7f49e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8041020 con 0x7f49f8070fd0 2024-04-25T01:02:04.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e8007820 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8036040 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f80ac2e0 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.593+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f80ad020 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.594+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f8075550 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.595+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49e8044020 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.595+0000 7f49dffff640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f49e406be30 0x7f49e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.595+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f49e80abdb0 con 0x7f49f8070fd0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.595+0000 7f49f67fc640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f49e406be30 0x7f49e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.596+0000 7f49f67fc640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f49e406be30 0x7f49e406e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49f80713b0 tx=0x7f49ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.599+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49e804b030 con 0x7f49f8070fd0 2024-04-25T01:02:04.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.772+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f49f80c61a0 con 0x7f49f8070fd0 2024-04-25T01:02:04.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.772+0000 7f49dffff640 1 -- 172.21.15.129:0/2306383633 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f49e807a4b0 con 0x7f49f8070fd0 2024-04-25T01:02:04.773 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f49e406be30 msgr2=0x7f49e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f49e406be30 0x7f49e406e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49f80713b0 tx=0x7f49ec008040 comp rx=0 tx=0).stop 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 msgr2=0x7f49f814f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 0x7f49f814f9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009f70 tx=0x7f49e8004a40 comp rx=0 tx=0).stop 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 shutdown_connections 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f49e406be30 0x7f49e406e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49f80c4430 0x7f49f80acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49f8156460 0x7f49f81eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 --2- 172.21.15.129:0/2306383633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49f8070fd0 0x7f49f814f9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 >> 172.21.15.129:0/2306383633 conn(0x7f49f8066ea0 msgr2=0x7f49f80c8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 shutdown_connections 2024-04-25T01:02:04.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.774+0000 7f49fda53640 1 -- 172.21.15.129:0/2306383633 wait complete. 2024-04-25T01:02:04.785 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:02:04.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.881+0000 7f0b145a9640 1 Processor -- start 2024-04-25T01:02:04.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.881+0000 7f0b145a9640 1 -- start start 2024-04-25T01:02:04.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b145a9640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0b0c152fa0 0x7f0b0c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b145a9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0b0c1561e0 0x7f0b0c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b145a9640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0c9860 con 0x7f0b0c0c5520 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b145a9640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0c0c5a70 con 0x7f0b0c1561e0 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b145a9640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0c0c5d50 con 0x7f0b0c152fa0 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b12b46640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f0b0c1561e0 0x7f0b0c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b11b44640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f0b0c0c5520 0x7f0b0c0c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44884/0 (socket says 172.21.15.129:44884) 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.882+0000 7f0b11b44640 1 -- 172.21.15.129:0/1108733793 learned_addr learned my addr 172.21.15.129:0/1108733793 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b12345640 1 --2- 172.21.15.129:0/1108733793 >> v2:172.21.15.171:3300/0 conn(0x7f0b0c152fa0 0x7f0b0c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b12b46640 1 -- 172.21.15.129:0/1108733793 >> v2:172.21.15.171:3300/0 conn(0x7f0b0c152fa0 msgr2=0x7f0b0c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b12b46640 1 --2- 172.21.15.129:0/1108733793 >> v2:172.21.15.171:3300/0 conn(0x7f0b0c152fa0 0x7f0b0c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b12b46640 1 -- 172.21.15.129:0/1108733793 >> v1:172.21.15.129:6789/0 conn(0x7f0b0c0c5520 legacy=0x7f0b0c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b12b46640 1 -- 172.21.15.129:0/1108733793 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c0c7650 con 0x7f0b0c1561e0 2024-04-25T01:02:04.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b11343640 1 -- 172.21.15.129:0/1108733793 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200239588 0 0) 0x7f0b08001110 con 0x7f0b0c0c5520 2024-04-25T01:02:04.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.883+0000 7f0b12b46640 1 --2- 172.21.15.129:0/1108733793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c0c8cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c08c590 tx=0x7f0b0402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645788b54c43d27d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b11343640 1 -- 172.21.15.129:0/1108733793 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04007740 con 0x7f0b0c1561e0 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b11343640 1 -- 172.21.15.129:0/1108733793 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b040078e0 con 0x7f0b0c1561e0 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b11343640 1 -- 172.21.15.129:0/1108733793 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04007cb0 con 0x7f0b0c1561e0 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 -- 172.21.15.129:0/1108733793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 msgr2=0x7f0b0c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 --2- 172.21.15.129:0/1108733793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c0c8cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c08c590 tx=0x7f0b0402a560 comp rx=0 tx=0).stop 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 -- 172.21.15.129:0/1108733793 shutdown_connections 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 --2- 172.21.15.129:0/1108733793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c0c8cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 --2- 172.21.15.129:0/1108733793 >> v2:172.21.15.171:3300/0 conn(0x7f0b0c152fa0 0x7f0b0c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 -- 172.21.15.129:0/1108733793 >> 172.21.15.129:0/1108733793 conn(0x7f0b0c06c400 msgr2=0x7f0b0c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:04.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 -- 172.21.15.129:0/1108733793 shutdown_connections 2024-04-25T01:02:04.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.884+0000 7f0b145a9640 1 -- 172.21.15.129:0/1108733793 wait complete. 2024-04-25T01:02:04.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.885+0000 7f0b145a9640 1 Processor -- start 2024-04-25T01:02:04.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.885+0000 7f0b145a9640 1 -- start start 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b145a9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c0c5520 0x7f0b0c14f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b145a9640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0b0c152fa0 0x7f0b0c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b145a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c1ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b145a9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c0c9200 con 0x7f0b0c1561e0 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b145a9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c0c6030 con 0x7f0b0c0c5520 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b145a9640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0c0c9380 con 0x7f0b0c152fa0 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c1ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c1ee670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39014/0 (socket says 172.21.15.129:39014) 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 -- 172.21.15.129:0/2860948708 learned_addr learned my addr 172.21.15.129:0/2860948708 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b11b44640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0b0c152fa0 0x7f0b0c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12345640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c0c5520 0x7f0b0c14f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 -- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0b0c152fa0 msgr2=0x7f0b0c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0b0c152fa0 0x7f0b0c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 -- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c0c5520 msgr2=0x7f0b0c14f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c0c5520 0x7f0b0c14f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 -- 172.21.15.129:0/2860948708 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c0c41f0 con 0x7f0b0c1561e0 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.886+0000 7f0b12b46640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c1ee670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009e50 tx=0x7f0b04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.887+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04040070 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.887+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b04031bf0 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.887+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c157710 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.887+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04009450 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.887+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1579b0 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.888+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c075550 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.888+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b04039040 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.888+0000 7f0afeffd640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0af006be30 0x7f0af006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.888+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0b040ac9a0 con 0x7f0b0c1561e0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.889+0000 7f0b12345640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0af006be30 0x7f0af006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:04.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.889+0000 7f0b12345640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0af006be30 0x7f0af006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c1556e0 tx=0x7f0b00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:04.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:04.893+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0b0407b1c0 con 0x7f0b0c1561e0 2024-04-25T01:02:05.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.101+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b0c1ee3d0 con 0x7f0b0c1561e0 2024-04-25T01:02:05.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.102+0000 7f0afeffd640 1 -- 172.21.15.129:0/2860948708 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f0b0407aac0 con 0x7f0b0c1561e0 2024-04-25T01:02:05.102 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:05.103 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:54.802314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:02:05.103 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-25T01:02:05.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0af006be30 msgr2=0x7f0af006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0af006be30 0x7f0af006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c1556e0 tx=0x7f0b00008040 comp rx=0 tx=0).stop 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 msgr2=0x7f0b0c1ee670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c1ee670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009e50 tx=0x7f0b04004a40 comp rx=0 tx=0).stop 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 shutdown_connections 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f0af006be30 0x7f0af006e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0b0c152fa0 0x7f0b0c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b0c1561e0 0x7f0b0c1ee670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 --2- 172.21.15.129:0/2860948708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b0c0c5520 0x7f0b0c14f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 >> 172.21.15.129:0/2860948708 conn(0x7f0b0c06c400 msgr2=0x7f0b0c0ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 shutdown_connections 2024-04-25T01:02:05.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.104+0000 7f0b145a9640 1 -- 172.21.15.129:0/2860948708 wait complete. 2024-04-25T01:02:05.116 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:02:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.212+0000 7fdcf88bd640 1 Processor -- start 2024-04-25T01:02:05.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.212+0000 7fdcf88bd640 1 -- start start 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf88bd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf40c6190 0x7fdcf40c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf88bd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcf406d270 0x7fdcf40abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf88bd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdcf40c2aa0 0x7fdcf41566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf88bd640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf41530e0 con 0x7fdcf406d270 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf88bd640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40c2f60 con 0x7fdcf40c2aa0 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf88bd640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf4153360 con 0x7fdcf40c6190 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf1d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcf406d270 0x7fdcf40abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf1d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcf406d270 0x7fdcf40abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39020/0 (socket says 172.21.15.129:39020) 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf1d9b640 1 -- 172.21.15.129:0/1740224627 learned_addr learned my addr 172.21.15.129:0/1740224627 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf2d9d640 1 --2- 172.21.15.129:0/1740224627 >> v2:172.21.15.171:3300/0 conn(0x7fdcf40c2aa0 0x7fdcf41566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.213+0000 7fdcf259c640 1 --2- 172.21.15.129:0/1740224627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf40c6190 0x7fdcf40c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf1d9b640 1 -- 172.21.15.129:0/1740224627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf40c6190 msgr2=0x7fdcf40c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf1d9b640 1 --2- 172.21.15.129:0/1740224627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf40c6190 0x7fdcf40c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf1d9b640 1 -- 172.21.15.129:0/1740224627 >> v2:172.21.15.171:3300/0 conn(0x7fdcf40c2aa0 msgr2=0x7fdcf41566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf1d9b640 1 --2- 172.21.15.129:0/1740224627 >> v2:172.21.15.171:3300/0 conn(0x7fdcf40c2aa0 0x7fdcf41566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf1d9b640 1 -- 172.21.15.129:0/1740224627 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf4153640 con 0x7fdcf406d270 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf1d9b640 1 --2- 172.21.15.129:0/1740224627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf406d270 0x7fdcf40abcc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40a6550 tx=0x7fdce802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c426ee14a2c9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf159a640 1 -- 172.21.15.129:0/1740224627 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8037070 con 0x7fdcf406d270 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf159a640 1 -- 172.21.15.129:0/1740224627 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce8032440 con 0x7fdcf406d270 2024-04-25T01:02:05.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.214+0000 7fdcf159a640 1 -- 172.21.15.129:0/1740224627 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8032810 con 0x7fdcf406d270 2024-04-25T01:02:05.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 -- 172.21.15.129:0/1740224627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf406d270 msgr2=0x7fdcf40abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/1740224627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf406d270 0x7fdcf40abcc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40a6550 tx=0x7fdce802a1e0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 -- 172.21.15.129:0/1740224627 shutdown_connections 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/1740224627 >> v2:172.21.15.171:3300/0 conn(0x7fdcf40c2aa0 0x7fdcf41566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/1740224627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf406d270 0x7fdcf40abcc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/1740224627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf40c6190 0x7fdcf40c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 -- 172.21.15.129:0/1740224627 >> 172.21.15.129:0/1740224627 conn(0x7fdcf40a86b0 msgr2=0x7fdcf40a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 -- 172.21.15.129:0/1740224627 shutdown_connections 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.215+0000 7fdcf88bd640 1 -- 172.21.15.129:0/1740224627 wait complete. 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.216+0000 7fdcf88bd640 1 Processor -- start 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.216+0000 7fdcf88bd640 1 -- start start 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf88bd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf406d270 0x7fdcf41e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf88bd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcf40c2aa0 0x7fdcf4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf88bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 0x7fdcf4148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf88bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4153fc0 con 0x7fdcf40c6190 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf88bd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4153e40 con 0x7fdcf40c2aa0 2024-04-25T01:02:05.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf88bd640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf40ac200 con 0x7fdcf406d270 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 0x7fdcf4148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 0x7fdcf4148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39036/0 (socket says 172.21.15.129:39036) 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 -- 172.21.15.129:0/3145754039 learned_addr learned my addr 172.21.15.129:0/3145754039 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf259c640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf406d270 0x7fdcf41e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf1d9b640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcf40c2aa0 0x7fdcf4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 -- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf406d270 msgr2=0x7fdcf41e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf406d270 0x7fdcf41e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 -- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcf40c2aa0 msgr2=0x7fdcf4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcf40c2aa0 0x7fdcf4156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 -- 172.21.15.129:0/3145754039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf41548d0 con 0x7fdcf40c6190 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdcf2d9d640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 0x7fdcf4148680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00eea0 tx=0x7fdcec00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.217+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec018020 con 0x7fdcf40c6190 2024-04-25T01:02:05.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.218+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec014400 con 0x7fdcf40c6190 2024-04-25T01:02:05.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.218+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf4154bb0 con 0x7fdcf40c6190 2024-04-25T01:02:05.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.218+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0147f0 con 0x7fdcf40c6190 2024-04-25T01:02:05.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.218+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf41e9de0 con 0x7fdcf40c6190 2024-04-25T01:02:05.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.219+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf40747d0 con 0x7fdcf40c6190 2024-04-25T01:02:05.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.219+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcec01b020 con 0x7fdcf40c6190 2024-04-25T01:02:05.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.219+0000 7fdce2ffd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdcd406be30 0x7fdcd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.220+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdcec08bbd0 con 0x7fdcf40c6190 2024-04-25T01:02:05.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.220+0000 7fdcf259c640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdcd406be30 0x7fdcd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.220+0000 7fdcf259c640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdcd406be30 0x7fdcd406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdcf41e6e20 tx=0x7fdce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:05.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.224+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdcf40747d0 con 0x7fdcf40c6190 2024-04-25T01:02:05.400 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.399+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcf40c20d0 con 0x7fdcf40c6190 2024-04-25T01:02:05.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.401+0000 7fdce2ffd640 1 -- 172.21.15.129:0/3145754039 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fdcf40c20d0 con 0x7fdcf40c6190 2024-04-25T01:02:05.401 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:05.402 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:01:37.763500+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:02:05.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdcd406be30 msgr2=0x7fdcd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdcd406be30 0x7fdcd406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdcf41e6e20 tx=0x7fdce4008040 comp rx=0 tx=0).stop 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 msgr2=0x7fdcf4148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 0x7fdcf4148680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00eea0 tx=0x7fdcec00ef90 comp rx=0 tx=0).stop 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 shutdown_connections 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdcd406be30 0x7fdcd406e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcf40c6190 0x7fdcf4148680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcf40c2aa0 0x7fdcf4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 --2- 172.21.15.129:0/3145754039 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcf406d270 0x7fdcf41e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 >> 172.21.15.129:0/3145754039 conn(0x7fdcf40a86b0 msgr2=0x7fdcf40a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 shutdown_connections 2024-04-25T01:02:05.404 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.403+0000 7fdcf88bd640 1 -- 172.21.15.129:0/3145754039 wait complete. 2024-04-25T01:02:05.415 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:02:05.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.511+0000 7f9d61d77640 1 Processor -- start 2024-04-25T01:02:05.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.511+0000 7f9d61d77640 1 -- start start 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d61d77640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9d5c0c5520 0x7f9d5c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d61d77640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0c9860 con 0x7f9d5c152fa0 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d61d77640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c0c5a70 con 0x7f9d5c1561e0 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d61d77640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c0c5d50 con 0x7f9d5c0c5520 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5affd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9d5c0c5520 0x7f9d5c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5b7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f9d5c152fa0 0x7f9d5c0c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44892/0 (socket says 172.21.15.129:44892) 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5b7fe640 1 -- 172.21.15.129:0/1508979477 learned_addr learned my addr 172.21.15.129:0/1508979477 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5affd640 1 --2- 172.21.15.129:0/1508979477 >> v2:172.21.15.129:3300/0 conn(0x7f9d5c0c5520 0x7f9d5c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39050/0 (socket says 172.21.15.129:39050) 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5affd640 1 -- 172.21.15.129:0/1508979477 >> v1:172.21.15.171:6789/0 conn(0x7f9d5c1561e0 legacy=0x7f9d5c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5affd640 1 -- 172.21.15.129:0/1508979477 >> v1:172.21.15.129:6789/0 conn(0x7f9d5c152fa0 legacy=0x7f9d5c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5affd640 1 -- 172.21.15.129:0/1508979477 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c0c7650 con 0x7f9d5c0c5520 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.512+0000 7f9d5a7fc640 1 -- 172.21.15.129:0/1508979477 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386941451 0 0) 0x7f9d4c001110 con 0x7f9d5c152fa0 2024-04-25T01:02:05.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.513+0000 7f9d5affd640 1 --2- 172.21.15.129:0/1508979477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c0ca2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d500097a0 tx=0x7f9d5002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de8d1f1e8bd3ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:05.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.513+0000 7f9d5a7fc640 1 -- 172.21.15.129:0/1508979477 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50037070 con 0x7f9d5c0c5520 2024-04-25T01:02:05.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.513+0000 7f9d5a7fc640 1 -- 172.21.15.129:0/1508979477 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50032440 con 0x7f9d5c0c5520 2024-04-25T01:02:05.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.513+0000 7f9d5a7fc640 1 -- 172.21.15.129:0/1508979477 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5002ae60 con 0x7f9d5c0c5520 2024-04-25T01:02:05.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 -- 172.21.15.129:0/1508979477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 msgr2=0x7f9d5c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1508979477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c0ca2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d500097a0 tx=0x7f9d5002a110 comp rx=0 tx=0).stop 2024-04-25T01:02:05.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 -- 172.21.15.129:0/1508979477 shutdown_connections 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1508979477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c0ca2e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 -- 172.21.15.129:0/1508979477 >> 172.21.15.129:0/1508979477 conn(0x7f9d5c06c400 msgr2=0x7f9d5c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 -- 172.21.15.129:0/1508979477 shutdown_connections 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.514+0000 7f9d61d77640 1 -- 172.21.15.129:0/1508979477 wait complete. 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.515+0000 7f9d61d77640 1 Processor -- start 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.515+0000 7f9d61d77640 1 -- start start 2024-04-25T01:02:05.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d61d77640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d61d77640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c152fa0 0x7f9d5c0c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d61d77640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c1561e0 0x7f9d5c0bf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d61d77640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c93d0 con 0x7f9d5c0c5520 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d61d77640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c6030 con 0x7f9d5c152fa0 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d61d77640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c9550 con 0x7f9d5c1561e0 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39062/0 (socket says 172.21.15.129:39062) 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 -- 172.21.15.129:0/1368938730 learned_addr learned my addr 172.21.15.129:0/1368938730 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5affd640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c152fa0 0x7f9d5c0c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5bfff640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c1561e0 0x7f9d5c0bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 -- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c1561e0 msgr2=0x7f9d5c0bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c1561e0 0x7f9d5c0bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 -- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c152fa0 msgr2=0x7f9d5c0c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c152fa0 0x7f9d5c0c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 -- 172.21.15.129:0/1368938730 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c0c39b0 con 0x7f9d5c0c5520 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d5b7fe640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c158f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00c780 tx=0x7f9d4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.516+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c018070 con 0x7f9d5c0c5520 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.517+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0072d0 con 0x7f9d5c0c5520 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.517+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0047e0 con 0x7f9d5c0c5520 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.517+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5c1581a0 con 0x7f9d5c0c5520 2024-04-25T01:02:05.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.517+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5c158760 con 0x7f9d5c0c5520 2024-04-25T01:02:05.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.518+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c06d480 con 0x7f9d5c0c5520 2024-04-25T01:02:05.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.518+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d4c004980 con 0x7f9d5c0c5520 2024-04-25T01:02:05.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.518+0000 7f9d43fff640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:05.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.519+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9d4c08a2e0 con 0x7f9d5c0c5520 2024-04-25T01:02:05.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.519+0000 7f9d5affd640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:05.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.519+0000 7f9d5affd640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d50034000 tx=0x7f9d5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:05.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.523+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d5c06d480 con 0x7f9d5c0c5520 2024-04-25T01:02:05.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.731+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d5c0c8770 con 0x7f9d5c0c5520 2024-04-25T01:02:05.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.732+0000 7f9d43fff640 1 -- 172.21.15.129:0/1368938730 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f9d5c0c8770 con 0x7f9d5c0c5520 2024-04-25T01:02:05.732 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:02:05.732 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:54.802314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:02:05.733 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-25T01:02:05.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.733+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 msgr2=0x7f9d4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.733+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d50034000 tx=0x7f9d5002e000 comp rx=0 tx=0).stop 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.733+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 msgr2=0x7f9d5c158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.733+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c158f50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00c780 tx=0x7f9d4c00cc40 comp rx=0 tx=0).stop 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 shutdown_connections 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9d4806be60 0x7f9d4806e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9d5c1561e0 0x7f9d5c0bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d5c152fa0 0x7f9d5c0c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 --2- 172.21.15.129:0/1368938730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d5c0c5520 0x7f9d5c158f50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 >> 172.21.15.129:0/1368938730 conn(0x7f9d5c06c400 msgr2=0x7f9d5c0abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 shutdown_connections 2024-04-25T01:02:05.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:05.734+0000 7f9d61d77640 1 -- 172.21.15.129:0/1368938730 wait complete. 2024-04-25T01:02:06.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:01:32.731107+0000', 'modified': '2024-04-25T01:01:54.802314+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.171:6837/1355919064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6835', 'nonce': 1355919064}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 1355919064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-04-25T01:02:06.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:02:06.148 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:02:06.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.247+0000 7f062a9e9640 1 Processor -- start 2024-04-25T01:02:06.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.247+0000 7f062a9e9640 1 -- start start 2024-04-25T01:02:06.248 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.247+0000 7f062a9e9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f062414fad0 0x7f06240abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.247+0000 7f062a9e9640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06240c4f10 con 0x7f062414f6f0 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.247+0000 7f062a9e9640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06240c5180 con 0x7f06240ac540 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.247+0000 7f062a9e9640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f0624155720 con 0x7f062414fad0 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0628f86640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f062414fad0 0x7f06240abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0623fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f062414f6f0 0x7f06240c4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44908/0 (socket says 172.21.15.129:44908) 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0623fff640 1 -- 172.21.15.129:0/3800737103 learned_addr learned my addr 172.21.15.129:0/3800737103 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0628f86640 1 -- 172.21.15.129:0/3800737103 >> v1:172.21.15.171:6789/0 conn(0x7f06240ac540 legacy=0x7f062406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0622ffd640 1 -- 172.21.15.129:0/3800737103 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549041490 0 0) 0x7f06240c4f10 con 0x7f062414f6f0 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0628f86640 1 -- 172.21.15.129:0/3800737103 >> v1:172.21.15.129:6789/0 conn(0x7f062414f6f0 legacy=0x7f06240c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.248+0000 7f0628f86640 1 -- 172.21.15.129:0/3800737103 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0624153010 con 0x7f062414fad0 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f0628f86640 1 --2- 172.21.15.129:0/3800737103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f062414fad0 0x7f06240abe50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a92e299f88fa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f0622ffd640 1 -- 172.21.15.129:0/3800737103 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c007740 con 0x7f062414fad0 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f0622ffd640 1 -- 172.21.15.129:0/3800737103 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c0078e0 con 0x7f062414fad0 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f0622ffd640 1 -- 172.21.15.129:0/3800737103 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c007cb0 con 0x7f062414fad0 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 -- 172.21.15.129:0/3800737103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f062414fad0 msgr2=0x7f06240abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 --2- 172.21.15.129:0/3800737103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f062414fad0 0x7f06240abe50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c02a660 comp rx=0 tx=0).stop 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 -- 172.21.15.129:0/3800737103 shutdown_connections 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 --2- 172.21.15.129:0/3800737103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f062414fad0 0x7f06240abe50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 -- 172.21.15.129:0/3800737103 >> 172.21.15.129:0/3800737103 conn(0x7f06240a86b0 msgr2=0x7f06240a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 -- 172.21.15.129:0/3800737103 shutdown_connections 2024-04-25T01:02:06.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.249+0000 7f062a9e9640 1 -- 172.21.15.129:0/3800737103 wait complete. 2024-04-25T01:02:06.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.250+0000 7f062a9e9640 1 Processor -- start 2024-04-25T01:02:06.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 -- start start 2024-04-25T01:02:06.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 0x7f06241dce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062414f6f0 0x7f06240c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062414fad0 0x7f06241e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0624155fa0 con 0x7f06240ac540 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0624155cc0 con 0x7f062414f6f0 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f062a9e9640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f062406ddd0 con 0x7f062414fad0 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 0x7f06241dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 0x7f06241dce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39088/0 (socket says 172.21.15.129:39088) 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.251+0000 7f0623fff640 1 -- 172.21.15.129:0/359851457 learned_addr learned my addr 172.21.15.129:0/359851457 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f06237fe640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062414f6f0 0x7f06240c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0628f86640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062414fad0 0x7f06241e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0623fff640 1 -- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062414fad0 msgr2=0x7f06241e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0623fff640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062414fad0 0x7f06241e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0623fff640 1 -- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062414f6f0 msgr2=0x7f06240c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0623fff640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062414f6f0 0x7f06240c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.254 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0623fff640 1 -- 172.21.15.129:0/359851457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062414e460 con 0x7f06240ac540 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f06237fe640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062414f6f0 0x7f06240c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0628f86640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062414fad0 0x7f06241e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0623fff640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 0x7f06241dce80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f061400c4a0 tx=0x7f061400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0620ff9640 1 -- 172.21.15.129:0/359851457 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614018070 con 0x7f06240ac540 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0620ff9640 1 -- 172.21.15.129:0/359851457 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06140074e0 con 0x7f06240ac540 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f0620ff9640 1 -- 172.21.15.129:0/359851457 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614004a20 con 0x7f06240ac540 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062414e740 con 0x7f06240ac540 2024-04-25T01:02:06.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.252+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06241db6c0 con 0x7f06240ac540 2024-04-25T01:02:06.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.253+0000 7f0620ff9640 1 -- 172.21.15.129:0/359851457 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f061401b070 con 0x7f06240ac540 2024-04-25T01:02:06.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.254+0000 7f0620ff9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f061006be30 0x7f061006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.254+0000 7f0620ff9640 1 -- 172.21.15.129:0/359851457 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f061408a470 con 0x7f06240ac540 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.254+0000 7f06237fe640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f061006be30 0x7f061006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.255+0000 7f06237fe640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f061006be30 0x7f061006e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06240c49e0 tx=0x7f0618009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.255+0000 7f062d7cc740 1 Processor -- start 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.255+0000 7f062d7cc740 1 -- start start 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5010dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5031510 0x5634f504d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f504a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f062d7cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5634f4f72f00 con 0x5634f504c560 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634f4f1b750 con 0x5634f504dd20 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x5634f4f6eec0 con 0x5634f5031510 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5010dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5010dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39092/0 (socket says 172.21.15.129:39092) 2024-04-25T01:02:06.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 -- 172.21.15.129:0/3023637954 learned_addr learned my addr 172.21.15.129:0/3023637954 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f06237fe640 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5031510 0x5634f504d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0628f86640 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f504a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 -- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5031510 msgr2=0x5634f504d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5031510 0x5634f504d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 -- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 msgr2=0x5634f504a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f504a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 -- 172.21.15.129:0/3023637954 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634f4c872d0 con 0x5634f504c560 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0623fff640 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5010dc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0614088690 tx=0x7f0614058cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b330bca1bca031d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0609ffb640 1 -- 172.21.15.129:0/3023637954 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614061050 con 0x5634f504c560 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0609ffb640 1 -- 172.21.15.129:0/3023637954 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0614056770 con 0x5634f504c560 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.256+0000 7f0609ffb640 1 -- 172.21.15.129:0/3023637954 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614060bc0 con 0x5634f504c560 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 -- 172.21.15.129:0/3023637954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 msgr2=0x5634f5010dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5010dc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0614088690 tx=0x7f0614058cb0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 -- 172.21.15.129:0/3023637954 shutdown_connections 2024-04-25T01:02:06.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5031510 0x5634f504d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f504a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3023637954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5010dc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 -- 172.21.15.129:0/3023637954 >> 172.21.15.129:0/3023637954 conn(0x5634f500c640 msgr2=0x5634f5030e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 -- 172.21.15.129:0/3023637954 shutdown_connections 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 -- 172.21.15.129:0/3023637954 wait complete. 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.257+0000 7f062d7cc740 1 -- start start 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 Processor -- start 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5033ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504dd20 0x5634f50324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f5045d30 0x5634f5044e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5634f500fd60 con 0x5634f504c560 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634f4f2ecf0 con 0x5634f5045d30 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x5634f504b390 con 0x5634f504dd20 2024-04-25T01:02:06.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f062d7cc740 10 client.0 _ll_register_callbacks cb 0x5634f4f2e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5033ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5033ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39098/0 (socket says 172.21.15.129:39098) 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.258+0000 7f0623fff640 1 -- 172.21.15.129:0/2408708967 learned_addr learned my addr 172.21.15.129:0/2408708967 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0628f86640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f5045d30 0x5634f5044e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f06237fe640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504dd20 0x5634f50324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0623fff640 1 -- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504dd20 msgr2=0x5634f50324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0623fff640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504dd20 0x5634f50324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0623fff640 1 -- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f5045d30 msgr2=0x5634f5044e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0623fff640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f5045d30 0x5634f5044e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0623fff640 1 -- 172.21.15.129:0/2408708967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634f4d516b0 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0623fff640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5033ba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f061400c470 tx=0x7f061400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d8ff7cc37aa38d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0602ffd640 1 -- 172.21.15.129:0/2408708967 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614075060 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0602ffd640 1 -- 172.21.15.129:0/2408708967 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0614004bc0 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0614056770 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0602ffd640 1 -- 172.21.15.129:0/2408708967 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f061407c050 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 10 client.4818 fetch_fsmap learned FSMap version 9 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5634f5049160 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0602ffd640 1 -- 172.21.15.129:0/2408708967 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0614056770 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f0602ffd640 1 -- 172.21.15.129:0/2408708967 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f06140772b0 con 0x5634f504c560 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 10 client.4818 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 20 client.4818 populate_metadata read hostname 'smithi129' 2024-04-25T01:02:06.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f5063c00 0x5634f5063fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 4 client.4818 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.259+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 --> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634f4f80440 con 0x5634f5063c00 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.260+0000 7f06237fe640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f5063c00 0x5634f5063fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.260+0000 7f06237fe640 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f5063c00 0x5634f5063fe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3c3d2906b66af6 server_cookie=6a1ec2806a6dafbe in_seq=0 out_seq=0 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.260+0000 7f0602ffd640 10 client.4818 ms_handle_connect on v2:172.21.15.171:6835/1355919064 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f0602ffd640 1 -- 172.21.15.129:0/2408708967 <== mds.0 v2:172.21.15.171:6835/1355919064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634f4f80440 con 0x5634f5063c00 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f0602ffd640 10 client.4818 handle_command_reply: tid=0 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 client.4818 shutdown 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 msgr2=0x5634f5033ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5033ba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f061400c470 tx=0x7f061400cf90 comp rx=0 tx=0).stop 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 shutdown_connections 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f5063c00 0x5634f5063fe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504dd20 0x5634f50324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f0602ffd640 0 client.4818 ms_handle_reset on v2:172.21.15.171:6835/1355919064 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f5045d30 0x5634f5044e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2408708967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504c560 0x5634f5033ba0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 >> 172.21.15.129:0/2408708967 conn(0x5634f5031510 msgr2=0x5634f5049fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 shutdown_connections 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 1 -- 172.21.15.129:0/2408708967 wait complete. 2024-04-25T01:02:06.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.261+0000 7f062d7cc740 20 client.4818 trim_cache size 0 max 16384 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 Processor -- start 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 -- start start 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 0x5634f504a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504c560 0x5634f5060260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f50471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06241f3b60 con 0x5634f5045d30 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0624072330 con 0x5634f504dd20 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f06241f3510 con 0x5634f504c560 2024-04-25T01:02:06.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 0x5634f504a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f0623fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 0x5634f504a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39102/0 (socket says 172.21.15.129:39102) 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.279+0000 7f0623fff640 1 -- 172.21.15.129:0/3758141114 learned_addr learned my addr 172.21.15.129:0/3758141114 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0623fff640 1 -- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504c560 msgr2=0x5634f5060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0623fff640 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504c560 0x5634f5060260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0623fff640 1 -- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 msgr2=0x5634f50471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0628f86640 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f50471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0623fff640 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f50471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0623fff640 1 -- 172.21.15.129:0/3758141114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634f504a700 con 0x5634f5045d30 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f0623fff640 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 0x5634f504a1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0614088740 tx=0x7f0614014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a277612e0dbf8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f06097fa640 1 -- 172.21.15.129:0/3758141114 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614056770 con 0x5634f5045d30 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f06097fa640 1 -- 172.21.15.129:0/3758141114 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06140772b0 con 0x5634f5045d30 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f06097fa640 1 -- 172.21.15.129:0/3758141114 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614004de0 con 0x5634f5045d30 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f062d7cc740 1 -- 172.21.15.129:0/3758141114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 msgr2=0x5634f504a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.280+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 0x5634f504a1c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0614088740 tx=0x7f0614014040 comp rx=0 tx=0).stop 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 -- 172.21.15.129:0/3758141114 shutdown_connections 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f504c560 0x5634f5060260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504dd20 0x5634f50471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 --2- 172.21.15.129:0/3758141114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f5045d30 0x5634f504a1c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 -- 172.21.15.129:0/3758141114 >> 172.21.15.129:0/3758141114 conn(0x5634f5031510 msgr2=0x5634f5005690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 -- 172.21.15.129:0/3758141114 shutdown_connections 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 -- 172.21.15.129:0/3758141114 wait complete. 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 -- start start 2024-04-25T01:02:06.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.281+0000 7f062d7cc740 1 Processor -- start 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5045d30 0x5634f5073520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504c560 0x5634f50caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 0x5634f50c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5634f500fd60 con 0x5634f504dd20 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5634f4f2ecf0 con 0x5634f504c560 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x5634f5045350 con 0x5634f5045d30 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f062d7cc740 10 client.0 _ll_register_callbacks cb 0x7f0618001380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 0x5634f50c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 0x5634f50c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39106/0 (socket says 172.21.15.129:39106) 2024-04-25T01:02:06.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 -- 172.21.15.129:0/2643782662 learned_addr learned my addr 172.21.15.129:0/2643782662 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 -- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5045d30 msgr2=0x5634f5073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5045d30 0x5634f5073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 -- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504c560 msgr2=0x5634f50caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504c560 0x5634f50caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 -- 172.21.15.129:0/2643782662 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634f505cf90 con 0x5634f504dd20 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0628f86640 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 0x5634f50c77d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f061c009eb0 tx=0x7f061c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d0409ba942da26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.282+0000 7f0602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f0602ffd640 1 -- 172.21.15.129:0/2643782662 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c007e90 con 0x5634f504dd20 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f0602ffd640 1 -- 172.21.15.129:0/2643782662 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f061c047660 con 0x5634f504dd20 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06140772b0 con 0x5634f504dd20 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f0602ffd640 1 -- 172.21.15.129:0/2643782662 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f061c054050 con 0x5634f504dd20 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 10 client.4824 fetch_fsmap learned FSMap version 9 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5634f50479d0 con 0x5634f504dd20 2024-04-25T01:02:06.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f0602ffd640 1 -- 172.21.15.129:0/2643782662 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06140772b0 con 0x5634f504dd20 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f0602ffd640 1 -- 172.21.15.129:0/2643782662 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f061c057020 con 0x5634f504dd20 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 10 client.4824 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 10 client.4824 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 20 client.4824 populate_metadata read hostname 'smithi129' 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f50cddc0 0x5634f50ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 4 client.4824 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 --> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] -- command(tid 0: {"prefix": "status"}) -- 0x5634f4f7a790 con 0x5634f50cddc0 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.283+0000 7f0623fff640 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f50cddc0 0x5634f50ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f0602ffd640 10 client.4824 ms_handle_connect on v2:172.21.15.171:6835/1355919064 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f0623fff640 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f50cddc0 0x5634f50ce1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40155e70f5822659 server_cookie=7d65d5435b92f8e2 in_seq=0 out_seq=0 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f0602ffd640 1 -- 172.21.15.129:0/2643782662 <== mds.0 v2:172.21.15.171:6835/1355919064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5634f4f7a790 con 0x5634f50cddc0 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f0602ffd640 10 client.4824 handle_command_reply: tid=0 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 client.4824 shutdown 2024-04-25T01:02:06.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 msgr2=0x5634f50c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 0x5634f50c77d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f061c009eb0 tx=0x7f061c02c000 comp rx=0 tx=0).stop 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 shutdown_connections 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:6835/1355919064,v1:172.21.15.171:6837/1355919064] conn(0x5634f50cddc0 0x5634f50ce1a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634f504dd20 0x5634f50c77d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5634f504c560 0x5634f50caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 --2- 172.21.15.129:0/2643782662 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x5634f5045d30 0x5634f5073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 >> 172.21.15.129:0/2643782662 conn(0x5634f5031510 msgr2=0x5634f5060260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f0602ffd640 0 client.4824 ms_handle_reset on v2:172.21.15.171:6835/1355919064 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 shutdown_connections 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 1 -- 172.21.15.129:0/2643782662 wait complete. 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.284+0000 7f062d7cc740 20 client.4824 trim_cache size 0 max 16384 2024-04-25T01:02:06.291 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "4b43d0d1-0ff7-4061-ba99-dacfcc159ca4", 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4294, 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 32.898219765, 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 9, 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 25, 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 40.044047665999997 2024-04-25T01:02:06.292 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:02:06.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f061006be30 msgr2=0x7f061006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f061006be30 0x7f061006e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06240c49e0 tx=0x7f0618009040 comp rx=0 tx=0).stop 2024-04-25T01:02:06.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 msgr2=0x7f06241dce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 0x7f06241dce80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f061400c4a0 tx=0x7f061400c960 comp rx=0 tx=0).stop 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 shutdown_connections 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f061006be30 0x7f061006e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f062414fad0 0x7f06241e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f062414f6f0 0x7f06240c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 --2- 172.21.15.129:0/359851457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06240ac540 0x7f06241dce80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 >> 172.21.15.129:0/359851457 conn(0x7f06240a86b0 msgr2=0x7f062406c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 shutdown_connections 2024-04-25T01:02:06.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.285+0000 7f062a9e9640 1 -- 172.21.15.129:0/359851457 wait complete. 2024-04-25T01:02:06.296 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T01:02:06.305 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T01:02:06.305 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T01:02:06.306 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T01:02:06.306 INFO:teuthology.run_tasks:Running task parallel... 2024-04-25T01:02:06.313 INFO:teuthology.task.parallel:starting parallel... 2024-04-25T01:02:06.314 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-25T01:02:06.314 INFO:teuthology.task.sequential:In sequential, running task ceph-fuse... 2024-04-25T01:02:06.315 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T01:02:06.315 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T01:02:06.315 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T01:02:06.316 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:02:06.316 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:06.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.316 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T01:02:06.316 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-25T01:02:06.316 INFO:teuthology.task.sequential:In sequential, running task kclient... 2024-04-25T01:02:06.320 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T01:02:06.320 INFO:tasks.kclient:config is {'client.1': {}} 2024-04-25T01:02:06.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.320 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-04-25T01:02:06.335 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T01:02:06.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.336 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:02:06.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-04-25T01:02:06.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.400 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-04-25T01:02:06.408 INFO:teuthology.orchestra.run.smithi171.stderr:Cannot find device "ceph-brx" 2024-04-25T01:02:06.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:02:06.409 INFO:tasks.kclient:client.1 config is {} 2024-04-25T01:02:06.409 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:02:06.410 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:06.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:02:06.515 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T01:02:06.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:02:06.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.540+0000 7f343d86b640 1 Processor -- start 2024-04-25T01:02:06.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.540+0000 7f343d86b640 1 -- start start 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f343d86b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34380ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f343d86b640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34380c9860 con 0x7f3438152fa0 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f343d86b640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3438156690 con 0x7f34381561e0 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f343d86b640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f34380c5a70 con 0x7f34380c5520 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3436ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f3438152fa0 0x7f34380c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44922/0 (socket says 172.21.15.129:44922) 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3436ffd640 1 -- 172.21.15.129:0/2878602806 learned_addr learned my addr 172.21.15.129:0/2878602806 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f34367fc640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34380ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536063217 0 0) 0x7f3428001110 con 0x7f3438152fa0 2024-04-25T01:02:06.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3420003520 con 0x7f3438152fa0 2024-04-25T01:02:06.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643322614 0 0) 0x7f3438156690 con 0x7f34381561e0 2024-04-25T01:02:06.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3428001110 con 0x7f34381561e0 2024-04-25T01:02:06.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798026748 0 0) 0x7f3420003520 con 0x7f3438152fa0 2024-04-25T01:02:06.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 msgr2=0x7f34380ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34380ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 >> v1:172.21.15.171:6789/0 conn(0x7f34381561e0 legacy=0x7f34380c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34380c5d50 con 0x7f3438152fa0 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f3428001840 con 0x7f3438152fa0 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f34367fc640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34380ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 >> v1:172.21.15.129:6789/0 conn(0x7f3438152fa0 legacy=0x7f34380c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420003b70 0x7f3420003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3420004800 0x7f3420004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34200052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34380c5a70 con 0x7f3420004800 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f342000bba0 con 0x7f3420003b70 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f342000bda0 con 0x7f34380c5520 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f34367fc640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3420004800 0x7f3420004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.541+0000 7f3436ffd640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420003b70 0x7f3420003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34377fe640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34200052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34367fc640 1 -- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 msgr2=0x7f34200052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34367fc640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380c5520 0x7f34200052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34367fc640 1 -- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420003b70 msgr2=0x7f3420003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34367fc640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420003b70 0x7f3420003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34367fc640 1 -- 172.21.15.129:0/2878602806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3420003520 con 0x7f3420004800 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.542+0000 7f34367fc640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3420004800 0x7f3420004c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f342c009c60 tx=0x7f3420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e48db8ec96b6e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c009860 con 0x7f3420004800 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f342c004020 con 0x7f3420004800 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f3435ffb640 1 -- 172.21.15.129:0/2878602806 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0043f0 con 0x7f3420004800 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f343d86b640 1 -- 172.21.15.129:0/2878602806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3420004800 msgr2=0x7f3420004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f343d86b640 1 --2- 172.21.15.129:0/2878602806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3420004800 0x7f3420004c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f342c009c60 tx=0x7f3420005830 comp rx=0 tx=0).stop 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f34377fe640 1 -- 172.21.15.129:0/2878602806 reap_dead start 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f343d86b640 1 -- 172.21.15.129:0/2878602806 shutdown_connections 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f343d86b640 1 -- 172.21.15.129:0/2878602806 >> 172.21.15.129:0/2878602806 conn(0x7f343806c400 msgr2=0x7f343806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f343d86b640 1 -- 172.21.15.129:0/2878602806 shutdown_connections 2024-04-25T01:02:06.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.543+0000 7f343d86b640 1 -- 172.21.15.129:0/2878602806 wait complete. 2024-04-25T01:02:06.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.545+0000 7f343d86b640 1 Processor -- start 2024-04-25T01:02:06.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.545+0000 7f343d86b640 1 -- start start 2024-04-25T01:02:06.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f343d86b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420004800 0x7f34380c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f343d86b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380be0b0 0x7f34380acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f343d86b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 0x7f34380bf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f343d86b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34380c93d0 con 0x7f34380bec00 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f343d86b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34380c6030 con 0x7f3420004800 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f343d86b640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f34380c9550 con 0x7f34380be0b0 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 0x7f34380bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 0x7f34380bf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39126/0 (socket says 172.21.15.129:39126) 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 -- 172.21.15.129:0/3075928364 learned_addr learned my addr 172.21.15.129:0/3075928364 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34367fc640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380be0b0 0x7f34380acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f3436ffd640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420004800 0x7f34380c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 -- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380be0b0 msgr2=0x7f34380acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380be0b0 0x7f34380acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 -- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420004800 msgr2=0x7f34380c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420004800 0x7f34380c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34377fe640 1 -- 172.21.15.129:0/3075928364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34380ab080 con 0x7f34380bec00 2024-04-25T01:02:06.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f34367fc640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380be0b0 0x7f34380acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.546+0000 7f3436ffd640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420004800 0x7f34380c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.547+0000 7f34377fe640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 0x7f34380bf090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f34381565c0 tx=0x7f343000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.547+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430003b80 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.547+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3430018070 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.547+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34380ab360 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.547+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34300050d0 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.547+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34380bf5d0 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.548+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3438075550 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.548+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3430005270 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.549+0000 7f341ffff640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f342406be30 0x7f342406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.549+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f343008af90 con 0x7f34380bec00 2024-04-25T01:02:06.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.549+0000 7f3436ffd640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f342406be30 0x7f342406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.549+0000 7f3436ffd640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f342406be30 0x7f342406e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34280010e0 tx=0x7f3428001340 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.553+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3430058fc0 con 0x7f34380bec00 2024-04-25T01:02:06.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.638+0000 7f68beed2640 1 Processor -- start 2024-04-25T01:02:06.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.638+0000 7f68beed2640 1 -- start start 2024-04-25T01:02:06.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68beed2640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f68b8152fa0 0x7f68b80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68beed2640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b80c5520 0x7f68b81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68beed2640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b80c9860 con 0x7f68b80c7e50 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68beed2640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f68b8156720 con 0x7f68b8152fa0 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68beed2640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f68b80c5a70 con 0x7f68b80c5520 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f68b8152fa0 0x7f68b80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f68b8152fa0 0x7f68b80ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39140/0 (socket says 172.21.15.129:39140) 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bd46f640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f68b80c7e50 0x7f68b80c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:44934/0 (socket says 172.21.15.129:44934) 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 -- 172.21.15.129:0/2671775120 learned_addr learned my addr 172.21.15.129:0/2671775120 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68b7fff640 1 --2- 172.21.15.129:0/2671775120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b80c5520 0x7f68b81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 -- 172.21.15.129:0/2671775120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b80c5520 msgr2=0x7f68b81561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 --2- 172.21.15.129:0/2671775120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b80c5520 0x7f68b81561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 -- 172.21.15.129:0/2671775120 >> v1:172.21.15.129:6789/0 conn(0x7f68b80c7e50 legacy=0x7f68b80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68b77fe640 1 -- 172.21.15.129:0/2671775120 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1280362026 0 0) 0x7f68b0001490 con 0x7f68b80c7e50 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.639+0000 7f68bcc6e640 1 -- 172.21.15.129:0/2671775120 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b80c5d50 con 0x7f68b8152fa0 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.640+0000 7f68bcc6e640 1 --2- 172.21.15.129:0/2671775120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b8152fa0 0x7f68b80ca2e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009870 tx=0x7f68ac02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe1b1dbf541c26b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.640+0000 7f68b77fe640 1 -- 172.21.15.129:0/2671775120 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac037070 con 0x7f68b8152fa0 2024-04-25T01:02:06.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.640+0000 7f68b77fe640 1 -- 172.21.15.129:0/2671775120 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac032440 con 0x7f68b8152fa0 2024-04-25T01:02:06.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.640+0000 7f68b77fe640 1 -- 172.21.15.129:0/2671775120 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac032810 con 0x7f68b8152fa0 2024-04-25T01:02:06.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.640+0000 7f68beed2640 1 -- 172.21.15.129:0/2671775120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b8152fa0 msgr2=0x7f68b80ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.640+0000 7f68beed2640 1 --2- 172.21.15.129:0/2671775120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b8152fa0 0x7f68b80ca2e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009870 tx=0x7f68ac02a130 comp rx=0 tx=0).stop 2024-04-25T01:02:06.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.641+0000 7f68beed2640 1 -- 172.21.15.129:0/2671775120 shutdown_connections 2024-04-25T01:02:06.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.641+0000 7f68beed2640 1 --2- 172.21.15.129:0/2671775120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b80c5520 0x7f68b81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.641+0000 7f68beed2640 1 --2- 172.21.15.129:0/2671775120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b8152fa0 0x7f68b80ca2e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.641+0000 7f68beed2640 1 -- 172.21.15.129:0/2671775120 >> 172.21.15.129:0/2671775120 conn(0x7f68b806c400 msgr2=0x7f68b806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.641+0000 7f68beed2640 1 -- 172.21.15.129:0/2671775120 shutdown_connections 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.641+0000 7f68beed2640 1 -- 172.21.15.129:0/2671775120 wait complete. 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.642+0000 7f68beed2640 1 Processor -- start 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.642+0000 7f68beed2640 1 -- start start 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.642+0000 7f68beed2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 0x7f68b80c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68b80c7e50 0x7f68b81e2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b8152fa0 0x7f68b8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80c93d0 con 0x7f68b80c5520 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80c6030 con 0x7f68b80c7e50 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f68b80c9550 con 0x7f68b8152fa0 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 0x7f68b80c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 0x7f68b80c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39146/0 (socket says 172.21.15.129:39146) 2024-04-25T01:02:06.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 -- 172.21.15.129:0/768481923 learned_addr learned my addr 172.21.15.129:0/768481923 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bd46f640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b8152fa0 0x7f68b8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68b7fff640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68b80c7e50 0x7f68b81e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 -- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b8152fa0 msgr2=0x7f68b8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b8152fa0 0x7f68b8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 -- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68b80c7e50 msgr2=0x7f68b81e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68b80c7e50 0x7f68b81e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 -- 172.21.15.129:0/768481923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b80bf090 con 0x7f68b80c5520 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bd46f640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b8152fa0 0x7f68b8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68b7fff640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68b80c7e50 0x7f68b81e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68bcc6e640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 0x7f68b80c0bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f68ac02a170 tx=0x7f68ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac037040 con 0x7f68b80c5520 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac042070 con 0x7f68b80c5520 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b80bf370 con 0x7f68b80c5520 2024-04-25T01:02:06.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac030980 con 0x7f68b80c5520 2024-04-25T01:02:06.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.643+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b80bf930 con 0x7f68b80c5520 2024-04-25T01:02:06.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.644+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8075550 con 0x7f68b80c5520 2024-04-25T01:02:06.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.645+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68ac030b20 con 0x7f68b80c5520 2024-04-25T01:02:06.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.645+0000 7f68b57fa640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f689c06be60 0x7f689c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:02:06.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.646+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68ac0aba30 con 0x7f68b80c5520 2024-04-25T01:02:06.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.646+0000 7f68b7fff640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f689c06be60 0x7f689c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:02:06.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.646+0000 7f68b7fff640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f689c06be60 0x7f689c06e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68a4006fd0 tx=0x7f68a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:02:06.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.650+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68ac07a130 con 0x7f68b80c5520 2024-04-25T01:02:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.770+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f34380c8aa0 con 0x7f34380bec00 2024-04-25T01:02:06.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.770+0000 7f341ffff640 1 -- 172.21.15.129:0/3075928364 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f343005d2c0 con 0x7f34380bec00 2024-04-25T01:02:06.771 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:02:06.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f342406be30 msgr2=0x7f342406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f342406be30 0x7f342406e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34280010e0 tx=0x7f3428001340 comp rx=0 tx=0).stop 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 msgr2=0x7f34380bf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 0x7f34380bf090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f34381565c0 tx=0x7f343000ff70 comp rx=0 tx=0).stop 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 shutdown_connections 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f342406be30 0x7f342406e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34380be0b0 0x7f34380acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34380bec00 0x7f34380bf090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 --2- 172.21.15.129:0/3075928364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3420004800 0x7f34380c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.772+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 >> 172.21.15.129:0/3075928364 conn(0x7f343806c400 msgr2=0x7f343806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.773+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 shutdown_connections 2024-04-25T01:02:06.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.773+0000 7f343d86b640 1 -- 172.21.15.129:0/3075928364 wait complete. 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi171.front.sepia.ceph.com 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:02:06.784 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:02:06.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.785 DEBUG:teuthology.orchestra.run.smithi171:> ip addr 2024-04-25T01:02:06.804 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:02:06.804 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f0 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:02:06.805 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: altname enp7s0f1 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6c brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6d brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.21.15.171/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft 42702sec preferred_lft 42702sec 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::3211:deec:96b4:2241/64 scope link noprefixroute 2024-04-25T01:02:06.806 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:02:06.807 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:02:06.807 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:06.807 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-04-25T01:02:06.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:02:06.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr flush dev ceph-brx 2024-04-25T01:02:06.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set ceph-brx up 2024-04-25T01:02:06.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:02:06.808 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-04-25T01:02:06.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.863+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f68b8155120 con 0x7f68b80c5520 2024-04-25T01:02:06.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.863+0000 7f68b57fa640 1 -- 172.21.15.129:0/768481923 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f68ac079a30 con 0x7f68b80c5520 2024-04-25T01:02:06.864 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:02:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.865+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f689c06be60 msgr2=0x7f689c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.865+0000 7f68beed2640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f689c06be60 0x7f689c06e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68a4006fd0 tx=0x7f68a4008040 comp rx=0 tx=0).stop 2024-04-25T01:02:06.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.865+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 msgr2=0x7f68b80c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.865+0000 7f68beed2640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 0x7f68b80c0bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f68ac02a170 tx=0x7f68ac0046c0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 shutdown_connections 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f689c06be60 0x7f689c06e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f68b8152fa0 0x7f68b8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68b80c7e50 0x7f68b81e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 --2- 172.21.15.129:0/768481923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68b80c5520 0x7f68b80c0bf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 >> 172.21.15.129:0/768481923 conn(0x7f68b806c400 msgr2=0x7f68b8155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 shutdown_connections 2024-04-25T01:02:06.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06.866+0000 7f68beed2640 1 -- 172.21.15.129:0/768481923 wait complete. 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:02:06.877 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:02:06.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.878 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-04-25T01:02:06.893 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42692sec preferred_lft 42692sec 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::fb51:43e0:ef26:52be/64 scope link noprefixroute 2024-04-25T01:02:06.894 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:02:06.894 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:02:06.894 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:06.894 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:02:06.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:02:06.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-04-25T01:02:06.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-04-25T01:02:06.895 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:02:06.895 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:02:06.901 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:06.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:06.992 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:06.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:06.999 DEBUG:teuthology.orchestra.run.smithi171:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:02:07.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.023 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-04-25T01:02:07.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.024 DEBUG:teuthology.orchestra.run.smithi171:> route 2024-04-25T01:02:07.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.028 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:02:07.050 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-04-25T01:02:07.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.051 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-04-25T01:02:07.093 INFO:teuthology.orchestra.run.smithi171.stdout:Kernel IP routing table 2024-04-25T01:02:07.093 INFO:teuthology.orchestra.run.smithi171.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:02:07.093 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:02:07.093 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:02:07.093 INFO:teuthology.orchestra.run.smithi171.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:02:07.093 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.093 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-04-25T01:02:07.093 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:02:07.093 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:02:07.094 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:02:07.094 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-04-25T01:02:07.107 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-04-25T01:02:07.107 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:02:07.107 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:02:07.107 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:02:07.107 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:02:07.107 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.107 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:02:07.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:02:07.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:02:07.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:02:07.108 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:02:07.181 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.230 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T01:02:07.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.244 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-04-25T01:02:07.299 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.300 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:02:07.300 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:02:07.300 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T01:02:07.300 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:02:07.374 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.381 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-04-25T01:02:07.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.396 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list-id 2024-04-25T01:02:07.408 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.415 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T01:02:07.415 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.415 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:02:07.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T01:02:07.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:02:07.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T01:02:07.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T01:02:07.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T01:02:07.416 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:02:07.451 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.451 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-04-25T01:02:07.451 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:02:07.452 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T01:02:07.452 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-04-25T01:02:07.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.524 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.524 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-04-25T01:02:07.524 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-04-25T01:02:07.524 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:02:07.524 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-04-25T01:02:07.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.566 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.573 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T01:02:07.573 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.573 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-04-25T01:02:07.573 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T01:02:07.573 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:02:07.573 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T01:02:07.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T01:02:07.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T01:02:07.574 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-04-25T01:02:07.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.597 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T01:02:07.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:02:07.598 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:07.612 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:02:07.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:02:07.612 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:07.623 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.667 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T01:02:07.707 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.713 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:02:07.713 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-04-25T01:02:07.713 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set brx.0 up 2024-04-25T01:02:07.713 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:02:07.714 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-04-25T01:02:07.731 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:02:07.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:07.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:07.787 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T01:02:07.787 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T01:02:07.787 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T01:02:07.787 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T01:02:07.787 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T01:02:07.787 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T01:02:07.788 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:02:07.789 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:02:07.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.790 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T01:02:07.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:02:07.791 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:07.805 INFO:teuthology.orchestra.run.smithi171.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:02:07.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:02:07.805 DEBUG:teuthology.orchestra.run.smithi171:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:07.842 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:02:07.842 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T01:02:07.860 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T01:02:07.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.860 DEBUG:teuthology.orchestra.run.smithi171:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T01:02:07.885 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-04-25T01:02:07.907 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:02:07.949 INFO:teuthology.orchestra.run.smithi171.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T01:02:07.949 INFO:teuthology.orchestra.run.smithi171.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-04-25T01:02:07.963 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:02:07.964 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:02:07.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:07.966 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-04-25T01:02:07.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:2024-04-25T01:02:07.962+0000 7f33af618480 -1 init, newargv = 0x56304dca00c0 newargc=15 2024-04-25T01:02:07.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[44877]: starting ceph client 2024-04-25T01:02:07.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[44877]: starting fuse 2024-04-25T01:02:07.981 INFO:teuthology.orchestra.run.smithi129.stdout:43 2024-04-25T01:02:07.982 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> import glob 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> import re 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> import os 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> import subprocess 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> def _find_admin_socket(client_name): 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi129:> files = glob.glob(asok_path) 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> # Given a non-glob path, it better be there 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> if "*" not in asok_path: 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> assert(len(files) == 1) 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> return files[0] 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> for f in files: 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> contents = proc_f.read() 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> if mountpoint in contents: 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> return f 2024-04-25T01:02:07.983 DEBUG:teuthology.orchestra.run.smithi129:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:02:07.984 DEBUG:teuthology.orchestra.run.smithi129:> 2024-04-25T01:02:07.984 DEBUG:teuthology.orchestra.run.smithi129:> print(_find_admin_socket("client.0")) 2024-04-25T01:02:07.984 DEBUG:teuthology.orchestra.run.smithi129:> ' 2024-04-25T01:02:08.053 INFO:teuthology.orchestra.run.smithi171.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T01:02:08.053 INFO:teuthology.orchestra.run.smithi171.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T01:02:08.053 INFO:teuthology.orchestra.run.smithi171.stdout:mount.ceph: resolved to: "172.21.15.129:3300,172.21.15.171:3300,172.21.15.171:3301" 2024-04-25T01:02:08.053 INFO:teuthology.orchestra.run.smithi171.stdout:mount.ceph: trying mount with new device syntax: 1@4b43d0d1-0ff7-4061-ba99-dacfcc159ca4.cephfs=/ 2024-04-25T01:02:08.054 INFO:teuthology.orchestra.run.smithi171.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.129:3300/172.21.15.171:3300/172.21.15.171:3301" will pass to kernel 2024-04-25T01:02:08.054 INFO:teuthology.orchestra.run.smithi171.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T01:02:08.054 INFO:teuthology.orchestra.run.smithi171.stdout: You just mounted a file system that supports labels which does not 2024-04-25T01:02:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T01:02:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T01:02:08.055 INFO:teuthology.orchestra.run.smithi171.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T01:02:08.056 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T01:02:08.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:08.056 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:08.079 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:02:08.098 INFO:teuthology.orchestra.run.smithi129.stdout:/var/run/ceph/ceph-client.0.44877.asok 2024-04-25T01:02:08.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:08.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44877.asok 2024-04-25T01:02:08.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:08.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44877.asok status 2024-04-25T01:02:08.167 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:02:08.168 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:08.174 INFO:teuthology.orchestra.run.smithi171.stdout:client4750 2024-04-25T01:02:08.174 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:02:08.175 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:02:08.190 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:02:08.240 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "metadata": { 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "entity_id": "0", 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": "smithi129", 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "pid": "44877", 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "root": "/" 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_count": 0, 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_pinned_count": 0, 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 4845, 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "inst": { 2024-04-25T01:02:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "name": { 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "client", 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "num": 4845 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3705080250 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:02:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3705080250 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "inst_str": "client.4845 192.168.0.1:0/3705080250", 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "addr_str": "192.168.0.1:0/3705080250", 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "inode_count": 1, 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_epoch": 9, 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch": 25, 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklisted": false, 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs" 2024-04-25T01:02:08.243 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:02:08.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:08.249 DEBUG:teuthology.orchestra.run.smithi129:> stat --file-system '--printf=%T 2024-04-25T01:02:08.249 DEBUG:teuthology.orchestra.run.smithi129:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:08.265 INFO:teuthology.orchestra.run.smithi129.stdout:fuseblk 2024-04-25T01:02:08.265 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:08.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:02:08.265 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:08.278 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:02:08.280 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:08.286 INFO:teuthology.orchestra.run.smithi171.stdout:client4750 2024-04-25T01:02:08.286 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:02:08.329 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-25T01:02:08.333 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:02:08.333 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T01:02:08.334 DEBUG:teuthology.orchestra.run.smithi129:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:08.334 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:02:08.335 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:08.341 INFO:teuthology.orchestra.run.smithi171.stdout:client4750 2024-04-25T01:02:08.341 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4b43d0d1-0ff7-4061-ba99-dacfcc159ca4.client4750/status') 2024-04-25T01:02:08.384 INFO:teuthology.orchestra.run.smithi129.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-25 01:01:33.387866020 +0000 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-25 01:01:43.081406942 +0000 2024-04-25T01:02:08.385 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: - 2024-04-25T01:02:08.386 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T01:02:08.386 DEBUG:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T01:02:08.389 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:02:08.399 INFO:teuthology.orchestra.run.smithi171.stdout:instance: client.4750 (3)192.168.0.1:0/808416584 2024-04-25T01:02:08.399 INFO:teuthology.orchestra.run.smithi171.stdout:blocklisted: false 2024-04-25T01:02:08.399 INFO:teuthology.orchestra.run.smithi171.stderr:0+1 records in 2024-04-25T01:02:08.399 INFO:teuthology.orchestra.run.smithi171.stderr:0+1 records out 2024-04-25T01:02:08.399 INFO:teuthology.orchestra.run.smithi171.stderr:68 bytes copied, 2.2127e-05 s, 3.1 MB/s 2024-04-25T01:02:08.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T01:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:02:08.406 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-25T01:02:08.406 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:02:08.407 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T01:02:08.407 DEBUG:teuthology.orchestra.run.smithi171:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 01:01:33.387866020 +0000 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 01:01:43.081406942 +0000 2024-04-25T01:02:08.422 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 01:01:33.387866020 +0000 2024-04-25T01:02:08.422 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-25T01:02:08.423 DEBUG:teuthology.orchestra.run.smithi171:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-25T01:02:08.453 INFO:tasks.workunit:timeout=3h 2024-04-25T01:02:08.453 INFO:tasks.workunit:cleanup=True 2024-04-25T01:02:08.453 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:02:08.490 INFO:tasks.workunit:timeout=3h 2024-04-25T01:02:08.490 INFO:tasks.workunit:cleanup=True 2024-04-25T01:02:08.491 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:02:08.509 INFO:tasks.workunit.client.0.smithi129.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T01:02:08.546 INFO:tasks.workunit.client.1.smithi171.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-25T01:02:28.166 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:02:28.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:02:58.322 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:02:58.328 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:03:15.894 INFO:tasks.workunit.client.0.smithi129.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: git switch -c 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr:Or undo this operation with: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: git switch - 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr: 2024-04-25T01:03:15.895 INFO:tasks.workunit.client.0.smithi129.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:03:15.897 DEBUG:teuthology.orchestra.run.smithi129:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T01:03:15.938 INFO:tasks.workunit.client.0.smithi129.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:03:15.940 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:03:15.940 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:03:16.463 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:03:16.521 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:03:16.561 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:03:16.562 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:03:16.563 INFO:tasks.workunit.client.0.smithi129.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:03:16.601 INFO:tasks.workunit.client.0.smithi129.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:03:16.604 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:03:16.604 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T01:03:16.670 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-25T01:03:16.671 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-25T01:03:16.672 DEBUG:teuthology.orchestra.run.smithi129:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-25T01:03:16.735 INFO:tasks.workunit.client.0.smithi129.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-25T01:03:16.837 INFO:tasks.workunit.client.0.smithi129.stdout: Iozone: Performance Test of File I/O 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Version $Revision: 3.494 $ 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Compiled for 64 bit mode. 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Build: linux-AMD64 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T01:03:16.838 INFO:tasks.workunit.client.0.smithi129.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Run began: Thu Apr 25 01:03:16 2024 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Include close in write timing 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Include fsync in write timing 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: File size set to 1048576 kB 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Record Size 16 kB 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Output is in kBytes/sec 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Processor cache line size set to 32 bytes. 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: File stride size set to 17 * record size. 2024-04-25T01:03:16.839 INFO:tasks.workunit.client.0.smithi129.stdout: Throughput test with 1 process 2024-04-25T01:03:16.840 INFO:tasks.workunit.client.0.smithi129.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-25T01:03:17.972 INFO:tasks.workunit.client.1.smithi171.stderr:Updating files: 94% (11761/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T01:03:18.786 INFO:tasks.workunit.client.1.smithi171.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:03:18.786 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.786 INFO:tasks.workunit.client.1.smithi171.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: git switch -c 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:Or undo this operation with: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: git switch - 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr: 2024-04-25T01:03:18.787 INFO:tasks.workunit.client.1.smithi171.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:03:18.790 DEBUG:teuthology.orchestra.run.smithi171:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-25T01:03:18.890 INFO:tasks.workunit.client.1.smithi171.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:03:18.892 INFO:tasks.workunit.client.1.smithi171.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T01:03:18.892 INFO:tasks.workunit.client.1.smithi171.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:03:19.515 INFO:tasks.workunit.client.1.smithi171.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:03:19.572 INFO:tasks.workunit.client.1.smithi171.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:03:19.611 INFO:tasks.workunit.client.1.smithi171.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T01:03:19.612 INFO:tasks.workunit.client.1.smithi171.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T01:03:19.612 INFO:tasks.workunit.client.1.smithi171.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:03:19.652 INFO:tasks.workunit.client.1.smithi171.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T01:03:19.656 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:03:19.656 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-25T01:03:19.672 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.1... 2024-04-25T01:03:19.673 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-25T01:03:19.673 DEBUG:teuthology.orchestra.run.smithi171:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh 2024-04-25T01:03:19.734 INFO:tasks.workunit.client.1.smithi171.stderr:+ dbench 1 2024-04-25T01:03:19.754 INFO:tasks.workunit.client.1.smithi171.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T01:03:19.755 INFO:tasks.workunit.client.1.smithi171.stdout: 2024-04-25T01:03:19.755 INFO:tasks.workunit.client.1.smithi171.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T01:03:19.755 INFO:tasks.workunit.client.1.smithi171.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-25T01:03:19.855 INFO:tasks.workunit.client.1.smithi171.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-25T01:03:19.855 INFO:tasks.workunit.client.1.smithi171.stdout:releasing clients 2024-04-25T01:03:20.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 3315 35.37 MB/sec warmup 1 sec latency 20.165 ms 2024-04-25T01:03:21.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 6120 24.01 MB/sec warmup 2 sec latency 12.389 ms 2024-04-25T01:03:22.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 9449 22.04 MB/sec warmup 3 sec latency 13.156 ms 2024-04-25T01:03:23.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 11856 20.10 MB/sec warmup 4 sec latency 38.059 ms 2024-04-25T01:03:24.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 15018 19.59 MB/sec warmup 5 sec latency 30.245 ms 2024-04-25T01:03:25.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 17988 19.37 MB/sec warmup 6 sec latency 50.274 ms 2024-04-25T01:03:26.398 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:26.398 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 initial writers = 109698.76 kB/sec 2024-04-25T01:03:26.398 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 initial writers = 109696.20 kB/sec 2024-04-25T01:03:26.398 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 109698.76 kB/sec 2024-04-25T01:03:26.399 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 109698.76 kB/sec 2024-04-25T01:03:26.399 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 109698.76 kB/sec 2024-04-25T01:03:26.399 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:03:26.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 21331 19.15 MB/sec warmup 7 sec latency 25.764 ms 2024-04-25T01:03:27.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 24894 19.15 MB/sec warmup 8 sec latency 23.798 ms 2024-04-25T01:03:28.416 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:03:28.418 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:03:28.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 28422 19.15 MB/sec warmup 9 sec latency 14.130 ms 2024-04-25T01:03:29.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 32080 19.17 MB/sec warmup 10 sec latency 26.651 ms 2024-04-25T01:03:30.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 35598 19.16 MB/sec warmup 11 sec latency 18.685 ms 2024-04-25T01:03:31.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 39263 19.25 MB/sec warmup 12 sec latency 20.559 ms 2024-04-25T01:03:32.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 42830 19.24 MB/sec warmup 13 sec latency 14.618 ms 2024-04-25T01:03:33.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 46367 19.23 MB/sec warmup 14 sec latency 19.761 ms 2024-04-25T01:03:34.856 INFO:tasks.workunit.client.1.smithi171.stdout: 1 49330 18.86 MB/sec warmup 15 sec latency 16.653 ms 2024-04-25T01:03:35.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 52286 18.55 MB/sec warmup 16 sec latency 16.132 ms 2024-04-25T01:03:36.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 54795 18.41 MB/sec warmup 17 sec latency 17.847 ms 2024-04-25T01:03:37.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 57707 18.21 MB/sec warmup 18 sec latency 18.811 ms 2024-04-25T01:03:38.856 INFO:tasks.workunit.client.1.smithi171.stdout: 1 61050 18.23 MB/sec warmup 19 sec latency 21.903 ms 2024-04-25T01:03:39.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 64350 18.26 MB/sec warmup 20 sec latency 14.450 ms 2024-04-25T01:03:40.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 67758 18.30 MB/sec warmup 21 sec latency 15.706 ms 2024-04-25T01:03:41.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 70953 18.26 MB/sec warmup 22 sec latency 16.248 ms 2024-04-25T01:03:42.320 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:42.320 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 rewriters = 117643.65 kB/sec 2024-04-25T01:03:42.321 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 rewriters = 117641.04 kB/sec 2024-04-25T01:03:42.321 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 117643.65 kB/sec 2024-04-25T01:03:42.321 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 117643.65 kB/sec 2024-04-25T01:03:42.321 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 117643.65 kB/sec 2024-04-25T01:03:42.321 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:03:42.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 74202 18.12 MB/sec warmup 23 sec latency 14.083 ms 2024-04-25T01:03:43.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 77438 18.10 MB/sec warmup 24 sec latency 21.363 ms 2024-04-25T01:03:44.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 80504 17.95 MB/sec warmup 25 sec latency 21.155 ms 2024-04-25T01:03:45.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 83836 17.95 MB/sec warmup 26 sec latency 12.879 ms 2024-04-25T01:03:46.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 86427 17.86 MB/sec warmup 27 sec latency 119.408 ms 2024-04-25T01:03:47.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 89890 17.90 MB/sec warmup 28 sec latency 15.468 ms 2024-04-25T01:03:48.341 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:48.341 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 readers = 7602345.00 kB/sec 2024-04-25T01:03:48.341 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 readers = 7593918.00 kB/sec 2024-04-25T01:03:48.341 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 7602345.00 kB/sec 2024-04-25T01:03:48.342 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 7602345.00 kB/sec 2024-04-25T01:03:48.342 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 7602345.00 kB/sec 2024-04-25T01:03:48.342 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:03:48.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 93272 17.90 MB/sec warmup 29 sec latency 28.472 ms 2024-04-25T01:03:49.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 96862 17.98 MB/sec warmup 30 sec latency 24.115 ms 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 re-readers = 7215675.00 kB/sec 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 re-readers = 7206016.27 kB/sec 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 7215675.00 kB/sec 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 7215675.00 kB/sec 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 7215675.00 kB/sec 2024-04-25T01:03:50.829 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:03:50.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 100429 18.01 MB/sec warmup 31 sec latency 20.092 ms 2024-04-25T01:03:51.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 100619 17.46 MB/sec warmup 32 sec latency 249.517 ms 2024-04-25T01:03:52.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 103518 17.44 MB/sec warmup 33 sec latency 98.565 ms 2024-04-25T01:03:53.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 106827 17.49 MB/sec warmup 34 sec latency 31.659 ms 2024-04-25T01:03:54.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 110338 17.54 MB/sec warmup 35 sec latency 24.840 ms 2024-04-25T01:03:55.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 113844 17.57 MB/sec warmup 36 sec latency 25.047 ms 2024-04-25T01:03:56.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 117085 17.58 MB/sec warmup 37 sec latency 58.265 ms 2024-04-25T01:03:57.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 119681 17.37 MB/sec warmup 38 sec latency 21.240 ms 2024-04-25T01:03:58.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 122882 17.36 MB/sec warmup 39 sec latency 41.202 ms 2024-04-25T01:03:59.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:03:59.134 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:03:59.384 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:59.384 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:59.384 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:59.384 INFO:tasks.workunit.client.0.smithi129.stdout:iozone test complete. 2024-04-25T01:03:59.384 INFO:tasks.workunit.client.0.smithi129.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-25T01:03:59.452 INFO:tasks.workunit.client.0.smithi129.stdout: Iozone: Performance Test of File I/O 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Version $Revision: 3.494 $ 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Compiled for 64 bit mode. 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Build: linux-AMD64 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T01:03:59.453 INFO:tasks.workunit.client.0.smithi129.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Run began: Thu Apr 25 01:03:59 2024 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Include close in write timing 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Include fsync in write timing 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: File size set to 1048576 kB 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Record Size 1024 kB 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Output is in kBytes/sec 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Processor cache line size set to 32 bytes. 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: File stride size set to 17 * record size. 2024-04-25T01:03:59.454 INFO:tasks.workunit.client.0.smithi129.stdout: Throughput test with 1 process 2024-04-25T01:03:59.455 INFO:tasks.workunit.client.0.smithi129.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-25T01:03:59.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 125009 17.27 MB/sec warmup 40 sec latency 56.767 ms 2024-04-25T01:04:00.856 INFO:tasks.workunit.client.1.smithi171.stdout: 1 126794 17.03 MB/sec warmup 41 sec latency 33.022 ms 2024-04-25T01:04:01.436 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:01.437 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 initial writers = 536779.12 kB/sec 2024-04-25T01:04:01.437 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 initial writers = 536693.18 kB/sec 2024-04-25T01:04:01.437 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 536779.12 kB/sec 2024-04-25T01:04:01.437 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 536779.12 kB/sec 2024-04-25T01:04:01.437 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 536779.12 kB/sec 2024-04-25T01:04:01.437 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:04:01.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 128530 16.90 MB/sec warmup 42 sec latency 26.089 ms 2024-04-25T01:04:02.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 131885 16.94 MB/sec warmup 43 sec latency 26.428 ms 2024-04-25T01:04:03.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 135361 16.99 MB/sec warmup 44 sec latency 16.833 ms 2024-04-25T01:04:04.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 137832 16.89 MB/sec warmup 45 sec latency 33.945 ms 2024-04-25T01:04:05.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 138815 16.67 MB/sec warmup 46 sec latency 25.737 ms 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 rewriters = 588436.06 kB/sec 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 rewriters = 588342.96 kB/sec 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 588436.06 kB/sec 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 588436.06 kB/sec 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 588436.06 kB/sec 2024-04-25T01:04:06.156 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:04:06.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 141755 16.62 MB/sec warmup 47 sec latency 45.029 ms 2024-04-25T01:04:07.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 144838 16.63 MB/sec warmup 48 sec latency 14.095 ms 2024-04-25T01:04:08.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 147958 16.59 MB/sec warmup 49 sec latency 20.631 ms 2024-04-25T01:04:09.856 INFO:tasks.workunit.client.1.smithi171.stdout: 1 150892 16.59 MB/sec warmup 50 sec latency 20.121 ms 2024-04-25T01:04:09.984 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:09.984 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 readers = 7424695.00 kB/sec 2024-04-25T01:04:09.984 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 readers = 7411745.00 kB/sec 2024-04-25T01:04:09.984 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 7424695.00 kB/sec 2024-04-25T01:04:09.985 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 7424695.00 kB/sec 2024-04-25T01:04:09.985 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 7424695.00 kB/sec 2024-04-25T01:04:09.985 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:04:10.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 153543 16.55 MB/sec warmup 51 sec latency 64.452 ms 2024-04-25T01:04:11.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 156074 16.55 MB/sec warmup 52 sec latency 169.066 ms 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 re-readers = 7310196.00 kB/sec 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 re-readers = 7297641.86 kB/sec 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 7310196.00 kB/sec 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 7310196.00 kB/sec 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 7310196.00 kB/sec 2024-04-25T01:04:12.470 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 1048576.00 kB 2024-04-25T01:04:12.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 158440 16.40 MB/sec warmup 53 sec latency 61.179 ms 2024-04-25T01:04:13.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 160765 16.36 MB/sec warmup 54 sec latency 24.929 ms 2024-04-25T01:04:14.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 164057 16.39 MB/sec warmup 55 sec latency 17.843 ms 2024-04-25T01:04:15.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 167535 16.43 MB/sec warmup 56 sec latency 21.778 ms 2024-04-25T01:04:16.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 171005 16.48 MB/sec warmup 57 sec latency 14.739 ms 2024-04-25T01:04:17.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 174504 16.52 MB/sec warmup 58 sec latency 18.750 ms 2024-04-25T01:04:18.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 177947 16.56 MB/sec warmup 59 sec latency 46.801 ms 2024-04-25T01:04:19.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 181251 16.60 MB/sec warmup 60 sec latency 22.333 ms 2024-04-25T01:04:20.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 184684 16.64 MB/sec warmup 61 sec latency 14.953 ms 2024-04-25T01:04:20.949 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:20.950 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:20.950 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:20.950 INFO:tasks.workunit.client.0.smithi129.stdout:iozone test complete. 2024-04-25T01:04:20.950 INFO:tasks.workunit.client.0.smithi129.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-25T01:04:21.016 INFO:tasks.workunit.client.0.smithi129.stdout: Iozone: Performance Test of File I/O 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Version $Revision: 3.494 $ 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Compiled for 64 bit mode. 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Build: linux-AMD64 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T01:04:21.017 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Run began: Thu Apr 25 01:04:20 2024 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Include close in write timing 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Include fsync in write timing 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: File size set to 10485760 kB 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Record Size 1024 kB 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Output is in kBytes/sec 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Processor cache line size set to 32 bytes. 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: File stride size set to 17 * record size. 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Throughput test with 1 process 2024-04-25T01:04:21.018 INFO:tasks.workunit.client.0.smithi129.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-25T01:04:21.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 186142 16.46 MB/sec warmup 62 sec latency 30.075 ms 2024-04-25T01:04:22.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 188265 16.42 MB/sec warmup 63 sec latency 23.555 ms 2024-04-25T01:04:23.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 189074 16.19 MB/sec warmup 64 sec latency 36.325 ms 2024-04-25T01:04:24.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 191215 16.12 MB/sec warmup 65 sec latency 32.536 ms 2024-04-25T01:04:25.855 INFO:tasks.workunit.client.1.smithi171.stdout: 1 192268 15.97 MB/sec warmup 66 sec latency 22.657 ms 2024-04-25T01:04:26.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 193894 15.83 MB/sec warmup 67 sec latency 37.273 ms 2024-04-25T01:04:27.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 195345 15.77 MB/sec warmup 68 sec latency 41.175 ms 2024-04-25T01:04:28.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 196251 15.58 MB/sec warmup 69 sec latency 31.027 ms 2024-04-25T01:04:29.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:04:29.367 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:04:29.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 198085 15.50 MB/sec warmup 70 sec latency 138.396 ms 2024-04-25T01:04:30.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 199320 15.38 MB/sec warmup 71 sec latency 26.760 ms 2024-04-25T01:04:31.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 200986 15.26 MB/sec warmup 72 sec latency 31.690 ms 2024-04-25T01:04:32.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 201398 15.11 MB/sec warmup 73 sec latency 256.213 ms 2024-04-25T01:04:33.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 202511 15.01 MB/sec warmup 74 sec latency 43.288 ms 2024-04-25T01:04:34.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 203745 14.88 MB/sec warmup 75 sec latency 29.863 ms 2024-04-25T01:04:35.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 205816 14.85 MB/sec warmup 76 sec latency 29.155 ms 2024-04-25T01:04:36.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 206603 14.68 MB/sec warmup 77 sec latency 22.689 ms 2024-04-25T01:04:37.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 208336 14.59 MB/sec warmup 78 sec latency 33.144 ms 2024-04-25T01:04:38.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 209724 14.55 MB/sec warmup 79 sec latency 22.616 ms 2024-04-25T01:04:39.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 210973 14.43 MB/sec warmup 80 sec latency 24.333 ms 2024-04-25T01:04:40.622 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:04:40.622 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 initial writers = 535704.31 kB/sec 2024-04-25T01:04:40.622 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 initial writers = 535696.62 kB/sec 2024-04-25T01:04:40.622 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 535704.31 kB/sec 2024-04-25T01:04:40.622 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 535704.31 kB/sec 2024-04-25T01:04:40.622 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 535704.31 kB/sec 2024-04-25T01:04:40.623 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 10485760.00 kB 2024-04-25T01:04:40.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 213141 14.42 MB/sec warmup 81 sec latency 25.709 ms 2024-04-25T01:04:41.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 216687 14.48 MB/sec warmup 82 sec latency 12.607 ms 2024-04-25T01:04:42.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 220175 14.53 MB/sec warmup 83 sec latency 13.283 ms 2024-04-25T01:04:43.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 223722 14.59 MB/sec warmup 84 sec latency 12.611 ms 2024-04-25T01:04:44.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 227165 14.63 MB/sec warmup 85 sec latency 18.006 ms 2024-04-25T01:04:45.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 230689 14.68 MB/sec warmup 86 sec latency 30.839 ms 2024-04-25T01:04:46.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 234158 14.73 MB/sec warmup 87 sec latency 14.003 ms 2024-04-25T01:04:47.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 237523 14.75 MB/sec warmup 88 sec latency 18.163 ms 2024-04-25T01:04:48.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 240675 14.77 MB/sec warmup 89 sec latency 19.239 ms 2024-04-25T01:04:49.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 243810 14.77 MB/sec warmup 90 sec latency 79.007 ms 2024-04-25T01:04:50.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 245660 14.74 MB/sec warmup 91 sec latency 40.239 ms 2024-04-25T01:04:51.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 247283 14.65 MB/sec warmup 92 sec latency 45.414 ms 2024-04-25T01:04:52.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 248585 14.62 MB/sec warmup 93 sec latency 26.553 ms 2024-04-25T01:04:53.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 248996 14.46 MB/sec warmup 94 sec latency 158.163 ms 2024-04-25T01:04:54.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 249243 14.32 MB/sec warmup 95 sec latency 74.850 ms 2024-04-25T01:04:55.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 251350 14.29 MB/sec warmup 96 sec latency 36.630 ms 2024-04-25T01:04:56.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 252546 14.21 MB/sec warmup 97 sec latency 70.993 ms 2024-04-25T01:04:57.857 INFO:tasks.workunit.client.1.smithi171.stdout: 1 254214 14.14 MB/sec warmup 98 sec latency 62.785 ms 2024-04-25T01:04:58.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 255796 14.12 MB/sec warmup 99 sec latency 34.842 ms 2024-04-25T01:04:59.762 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:04:59.765 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:04:59.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 256890 14.03 MB/sec warmup 100 sec latency 39.289 ms 2024-04-25T01:05:00.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 259186 14.03 MB/sec warmup 101 sec latency 27.784 ms 2024-04-25T01:05:01.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 259913 13.90 MB/sec warmup 102 sec latency 30.135 ms 2024-04-25T01:05:02.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 262164 13.89 MB/sec warmup 103 sec latency 28.158 ms 2024-04-25T01:05:03.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 263257 13.81 MB/sec warmup 104 sec latency 34.820 ms 2024-04-25T01:05:04.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 264991 13.75 MB/sec warmup 105 sec latency 40.046 ms 2024-04-25T01:05:05.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 266483 13.73 MB/sec warmup 106 sec latency 39.100 ms 2024-04-25T01:05:06.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 267921 13.65 MB/sec warmup 107 sec latency 22.980 ms 2024-04-25T01:05:07.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 269847 13.65 MB/sec warmup 108 sec latency 22.421 ms 2024-04-25T01:05:08.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 270823 13.54 MB/sec warmup 109 sec latency 25.513 ms 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 rewriters = 556875.06 kB/sec 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 rewriters = 556867.18 kB/sec 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 556875.06 kB/sec 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 556875.06 kB/sec 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 556875.06 kB/sec 2024-04-25T01:05:09.147 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 10485760.00 kB 2024-04-25T01:05:09.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 273604 13.58 MB/sec warmup 110 sec latency 62.555 ms 2024-04-25T01:05:10.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 277057 13.63 MB/sec warmup 111 sec latency 86.924 ms 2024-04-25T01:05:11.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 280490 13.67 MB/sec warmup 112 sec latency 20.251 ms 2024-04-25T01:05:12.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 283987 13.72 MB/sec warmup 113 sec latency 25.771 ms 2024-04-25T01:05:13.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 287419 13.76 MB/sec warmup 114 sec latency 18.138 ms 2024-04-25T01:05:14.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 288838 13.70 MB/sec warmup 115 sec latency 252.394 ms 2024-04-25T01:05:15.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 291410 13.70 MB/sec warmup 116 sec latency 52.200 ms 2024-04-25T01:05:16.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 294881 13.74 MB/sec warmup 117 sec latency 24.666 ms 2024-04-25T01:05:17.858 INFO:tasks.workunit.client.1.smithi171.stdout: 1 298445 13.79 MB/sec warmup 118 sec latency 16.278 ms 2024-04-25T01:05:18.859 INFO:tasks.workunit.client.1.smithi171.stdout: 1 301907 13.83 MB/sec warmup 119 sec latency 14.821 ms 2024-04-25T01:05:20.859 INFO:tasks.workunit.client.1.smithi171.stdout: 1 308905 19.12 MB/sec execute 1 sec latency 16.571 ms 2024-04-25T01:05:21.859 INFO:tasks.workunit.client.1.smithi171.stdout: 1 312289 18.61 MB/sec execute 2 sec latency 14.671 ms 2024-04-25T01:05:22.859 INFO:tasks.workunit.client.1.smithi171.stdout: 1 315822 18.77 MB/sec execute 3 sec latency 20.830 ms 2024-04-25T01:05:23.859 INFO:tasks.workunit.client.1.smithi171.stdout: 1 319336 18.84 MB/sec execute 4 sec latency 16.335 ms 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 readers = 7639185.50 kB/sec 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 readers = 7637872.57 kB/sec 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 7639185.50 kB/sec 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 7639185.50 kB/sec 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 7639185.50 kB/sec 2024-04-25T01:05:24.824 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 10485760.00 kB 2024-04-25T01:05:24.860 INFO:tasks.workunit.client.1.smithi171.stdout: 1 322622 18.26 MB/sec execute 5 sec latency 19.716 ms 2024-04-25T01:05:25.860 INFO:tasks.workunit.client.1.smithi171.stdout: 1 326302 18.88 MB/sec execute 6 sec latency 12.862 ms 2024-04-25T01:05:26.860 INFO:tasks.workunit.client.1.smithi171.stdout: 1 329512 18.31 MB/sec execute 7 sec latency 20.807 ms 2024-04-25T01:05:27.860 INFO:tasks.workunit.client.1.smithi171.stdout: 1 332645 18.14 MB/sec execute 8 sec latency 23.879 ms 2024-04-25T01:05:28.757 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:05:28.757 INFO:tasks.workunit.client.0.smithi129.stdout: Children see throughput for 1 re-readers = 7517907.50 kB/sec 2024-04-25T01:05:28.757 INFO:tasks.workunit.client.0.smithi129.stdout: Parent sees throughput for 1 re-readers = 7516701.17 kB/sec 2024-04-25T01:05:28.758 INFO:tasks.workunit.client.0.smithi129.stdout: Min throughput per process = 7517907.50 kB/sec 2024-04-25T01:05:28.758 INFO:tasks.workunit.client.0.smithi129.stdout: Max throughput per process = 7517907.50 kB/sec 2024-04-25T01:05:28.758 INFO:tasks.workunit.client.0.smithi129.stdout: Avg throughput per process = 7517907.50 kB/sec 2024-04-25T01:05:28.758 INFO:tasks.workunit.client.0.smithi129.stdout: Min xfer = 10485760.00 kB 2024-04-25T01:05:28.862 INFO:tasks.workunit.client.1.smithi171.stdout: 1 335774 17.79 MB/sec execute 9 sec latency 19.296 ms 2024-04-25T01:05:29.848 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:05:29.851 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:05:29.862 INFO:tasks.workunit.client.1.smithi171.stdout: 1 338587 17.75 MB/sec execute 10 sec latency 51.697 ms 2024-04-25T01:05:30.864 INFO:tasks.workunit.client.1.smithi171.stdout: 1 341405 17.47 MB/sec execute 11 sec latency 39.761 ms 2024-04-25T01:05:31.865 INFO:tasks.workunit.client.1.smithi171.stdout: 1 344875 17.59 MB/sec execute 12 sec latency 26.554 ms 2024-04-25T01:05:32.865 INFO:tasks.workunit.client.1.smithi171.stdout: 1 348423 17.70 MB/sec execute 13 sec latency 13.037 ms 2024-04-25T01:05:33.865 INFO:tasks.workunit.client.1.smithi171.stdout: 1 351958 17.80 MB/sec execute 14 sec latency 20.325 ms 2024-04-25T01:05:34.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 355377 17.87 MB/sec execute 15 sec latency 57.167 ms 2024-04-25T01:05:35.865 INFO:tasks.workunit.client.1.smithi171.stdout: 1 356349 17.07 MB/sec execute 16 sec latency 536.076 ms 2024-04-25T01:05:36.865 INFO:tasks.workunit.client.1.smithi171.stdout: 1 357907 16.54 MB/sec execute 17 sec latency 32.550 ms 2024-04-25T01:05:37.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 358868 15.94 MB/sec execute 18 sec latency 18.537 ms 2024-04-25T01:05:38.841 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:05:38.841 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:05:38.841 INFO:tasks.workunit.client.0.smithi129.stdout: 2024-04-25T01:05:38.841 INFO:tasks.workunit.client.0.smithi129.stdout:iozone test complete. 2024-04-25T01:05:38.842 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:05:38.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:05:38.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 359779 15.38 MB/sec execute 19 sec latency 28.828 ms 2024-04-25T01:05:38.892 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-25T01:05:38.892 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T01:05:39.254 DEBUG:teuthology.parallel:result is None 2024-04-25T01:05:39.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:05:39.286 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:05:39.286 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T01:05:39.286 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:05:39.286 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:05:39.341 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:05:39.341 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:05:39.396 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:05:39.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.542+0000 7ffa6999e640 1 Processor -- start 2024-04-25T01:05:39.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.542+0000 7ffa6999e640 1 -- start start 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa6999e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa640c75c0 0x7ffa6406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa6999e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa64153da0 con 0x7ffa64070e10 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa6999e640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa640c8350 con 0x7ffa640c4e50 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa6999e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ffa640c6990 con 0x7ffa640c75c0 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa62ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ffa64070e10 0x7ffa640aa6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55412/0 (socket says 172.21.15.129:55412) 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa62ffd640 1 -- 172.21.15.129:0/1475331732 learned_addr learned my addr 172.21.15.129:0/1475331732 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa637fe640 1 --2- 172.21.15.129:0/1475331732 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa640c75c0 0x7ffa6406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866511482 0 0) 0x7ffa64153da0 con 0x7ffa64070e10 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.543+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa50003520 con 0x7ffa64070e10 2024-04-25T01:05:39.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.544+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720288096 0 0) 0x7ffa640c8350 con 0x7ffa640c4e50 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.544+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa64153da0 con 0x7ffa640c4e50 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.544+0000 7ffa637fe640 1 -- 172.21.15.129:0/1475331732 >> v1:172.21.15.171:6789/0 conn(0x7ffa640c4e50 legacy=0x7ffa640c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.544+0000 7ffa637fe640 1 -- 172.21.15.129:0/1475331732 >> v1:172.21.15.129:6789/0 conn(0x7ffa64070e10 legacy=0x7ffa640aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.544+0000 7ffa637fe640 1 -- 172.21.15.129:0/1475331732 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa64153ab0 con 0x7ffa640c75c0 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.544+0000 7ffa637fe640 1 --2- 172.21.15.129:0/1475331732 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa640c75c0 0x7ffa6406e8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009dc0 tx=0x7ffa5c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6b3c52a2de5628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c007740 con 0x7ffa640c75c0 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c0078e0 con 0x7ffa640c75c0 2024-04-25T01:05:39.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa627fc640 1 -- 172.21.15.129:0/1475331732 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c007c90 con 0x7ffa640c75c0 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 -- 172.21.15.129:0/1475331732 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa640c75c0 msgr2=0x7ffa6406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 --2- 172.21.15.129:0/1475331732 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa640c75c0 0x7ffa6406e8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009dc0 tx=0x7ffa5c02a6d0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 -- 172.21.15.129:0/1475331732 shutdown_connections 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 --2- 172.21.15.129:0/1475331732 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa640c75c0 0x7ffa6406e8c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 -- 172.21.15.129:0/1475331732 >> 172.21.15.129:0/1475331732 conn(0x7ffa640c5490 msgr2=0x7ffa640c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 -- 172.21.15.129:0/1475331732 shutdown_connections 2024-04-25T01:05:39.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.545+0000 7ffa6999e640 1 -- 172.21.15.129:0/1475331732 wait complete. 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 Processor -- start 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 -- start start 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa64070e10 0x7ffa640ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 0x7ffa640c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa640c75c0 0x7ffa6406cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640aade0 con 0x7ffa640c4e50 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640c8630 con 0x7ffa640c75c0 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa6999e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ffa640aaf60 con 0x7ffa64070e10 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 0x7ffa640c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.547+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 0x7ffa640c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55500/0 (socket says 172.21.15.129:55500) 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa5bfff640 1 -- 172.21.15.129:0/362461916 learned_addr learned my addr 172.21.15.129:0/362461916 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa64070e10 0x7ffa640ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa637fe640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa640c75c0 0x7ffa6406cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 -- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa640c75c0 msgr2=0x7ffa6406cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa640c75c0 0x7ffa6406cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 -- 172.21.15.129:0/362461916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 msgr2=0x7ffa640c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 0x7ffa640c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 -- 172.21.15.129:0/362461916 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa641eaa40 con 0x7ffa64070e10 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa5bfff640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 0x7ffa640c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa62ffd640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa64070e10 0x7ffa640ab7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c390 tx=0x7ffa5400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54017070 con 0x7ffa64070e10 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa54005ce0 con 0x7ffa64070e10 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6406c220 con 0x7ffa64070e10 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540072d0 con 0x7ffa64070e10 2024-04-25T01:05:39.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.548+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6406c7e0 con 0x7ffa64070e10 2024-04-25T01:05:39.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.549+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa64075550 con 0x7ffa64070e10 2024-04-25T01:05:39.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.550+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa5401a070 con 0x7ffa64070e10 2024-04-25T01:05:39.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.550+0000 7ffa60ff9640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ffa4406be30 0x7ffa4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.550+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffa5408a4f0 con 0x7ffa64070e10 2024-04-25T01:05:39.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.550+0000 7ffa5bfff640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ffa4406be30 0x7ffa4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.551+0000 7ffa5bfff640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ffa4406be30 0x7ffa4406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffa640ab1e0 tx=0x7ffa4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:39.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.555+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa540583c0 con 0x7ffa64070e10 2024-04-25T01:05:39.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.740+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa640c7f50 con 0x7ffa64070e10 2024-04-25T01:05:39.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.742+0000 7ffa60ff9640 1 -- 172.21.15.129:0/362461916 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18891 (secure 0 0 0) 0x7ffa5405c6c0 con 0x7ffa64070e10 2024-04-25T01:05:39.738 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:05:39.738 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:02:13.734125+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:05:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ffa4406be30 msgr2=0x7ffa4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ffa4406be30 0x7ffa4406e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffa640ab1e0 tx=0x7ffa4c009040 comp rx=0 tx=0).stop 2024-04-25T01:05:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa64070e10 msgr2=0x7ffa640ab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa64070e10 0x7ffa640ab7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c390 tx=0x7ffa5400c850 comp rx=0 tx=0).stop 2024-04-25T01:05:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 shutdown_connections 2024-04-25T01:05:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ffa4406be30 0x7ffa4406e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa640c75c0 0x7ffa6406cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa640c4e50 0x7ffa640c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.741 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 --2- 172.21.15.129:0/362461916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa64070e10 0x7ffa640ab7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 >> 172.21.15.129:0/362461916 conn(0x7ffa640c5490 msgr2=0x7ffa64152510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:39.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 shutdown_connections 2024-04-25T01:05:39.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.744+0000 7ffa6999e640 1 -- 172.21.15.129:0/362461916 wait complete. 2024-04-25T01:05:39.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:05:39.753 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:05:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.864+0000 7f7ac0e1c640 1 Processor -- start 2024-04-25T01:05:39.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.864+0000 7f7ac0e1c640 1 -- start start 2024-04-25T01:05:39.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 362971 15.41 MB/sec execute 20 sec latency 15.863 ms 2024-04-25T01:05:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ac0e1c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ac0e1c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7abc159630 0x7f7abc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ac0e1c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7abc0c5c10 con 0x7f7abc156050 2024-04-25T01:05:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ac0e1c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc0ca4b0 con 0x7f7abc159630 2024-04-25T01:05:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ac0e1c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc0c5650 con 0x7f7abc0c8cc0 2024-04-25T01:05:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab9d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7abc159630 0x7f7abc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7abad9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f7abc156050 0x7f7abc0c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55416/0 (socket says 172.21.15.129:55416) 2024-04-25T01:05:39.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab9d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7abc159630 0x7f7abc0c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55504/0 (socket says 172.21.15.129:55504) 2024-04-25T01:05:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7abad9d640 1 -- 172.21.15.129:0/1691941104 learned_addr learned my addr 172.21.15.129:0/1691941104 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:05:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7aba59c640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/1691941104 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 msgr2=0x7f7abc0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab9d9b640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/1691941104 >> v1:172.21.15.129:6789/0 conn(0x7f7abc156050 legacy=0x7f7abc0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab959a640 1 -- 172.21.15.129:0/1691941104 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3015396813 0 0) 0x7f7abc0c5c10 con 0x7f7abc156050 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/1691941104 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc0c5930 con 0x7f7abc159630 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.865+0000 7f7aba59c640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.866+0000 7f7ab9d9b640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc159630 0x7f7abc0c9f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00097a0 tx=0x7f7ab002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc62e645106eef05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.866+0000 7f7ab959a640 1 -- 172.21.15.129:0/1691941104 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0037070 con 0x7f7abc159630 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.866+0000 7f7ab959a640 1 -- 172.21.15.129:0/1691941104 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab002ae50 con 0x7f7abc159630 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.866+0000 7f7ab959a640 1 -- 172.21.15.129:0/1691941104 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0032660 con 0x7f7abc159630 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/1691941104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc159630 msgr2=0x7f7abc0c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc159630 0x7f7abc0c9f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00097a0 tx=0x7f7ab002a100 comp rx=0 tx=0).stop 2024-04-25T01:05:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/1691941104 shutdown_connections 2024-04-25T01:05:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc159630 0x7f7abc0c9f70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/1691941104 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/1691941104 >> 172.21.15.129:0/1691941104 conn(0x7f7abc06c400 msgr2=0x7f7abc06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:39.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/1691941104 shutdown_connections 2024-04-25T01:05:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.867+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/1691941104 wait complete. 2024-04-25T01:05:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.868+0000 7f7ac0e1c640 1 Processor -- start 2024-04-25T01:05:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.868+0000 7f7ac0e1c640 1 -- start start 2024-04-25T01:05:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ac0e1c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ac0e1c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 0x7f7abc1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ac0e1c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc159630 0x7f7abc1eb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ac0e1c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc06d1f0 con 0x7f7abc156050 2024-04-25T01:05:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ac0e1c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc159a70 con 0x7f7abc159630 2024-04-25T01:05:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ac0e1c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc0c5f30 con 0x7f7abc0c8cc0 2024-04-25T01:05:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 0x7f7abc1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 0x7f7abc1581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55512/0 (socket says 172.21.15.129:55512) 2024-04-25T01:05:39.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7aba59c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/3246414561 learned_addr learned my addr 172.21.15.129:0/3246414561 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7abad9d640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc159630 0x7f7abc1eb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 msgr2=0x7f7abc0c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc159630 msgr2=0x7f7abc1eb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:39.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.869+0000 7f7ab9d9b640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc159630 0x7f7abc1eb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7ab9d9b640 1 -- 172.21.15.129:0/3246414561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc155280 con 0x7f7abc156050 2024-04-25T01:05:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7ab9d9b640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 0x7f7abc1581f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0000c00 tx=0x7f7ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0042070 con 0x7f7abc156050 2024-04-25T01:05:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab003e460 con 0x7f7abc156050 2024-04-25T01:05:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc1eeff0 con 0x7f7abc156050 2024-04-25T01:05:39.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0037040 con 0x7f7abc156050 2024-04-25T01:05:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.870+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc1ef2b0 con 0x7f7abc156050 2024-04-25T01:05:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.871+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc075550 con 0x7f7abc156050 2024-04-25T01:05:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.871+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ab0045070 con 0x7f7abc156050 2024-04-25T01:05:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.872+0000 7f7aaaffd640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:39.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.872+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7ab00ab7c0 con 0x7f7abc156050 2024-04-25T01:05:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.872+0000 7f7aba59c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.872+0000 7f7aba59c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009560 tx=0x7f7aac009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:39.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:39.876+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ab00797b0 con 0x7f7abc156050 2024-04-25T01:05:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.081+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7abc0c9b40 con 0x7f7abc156050 2024-04-25T01:05:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.081+0000 7f7aaaffd640 1 -- 172.21.15.129:0/3246414561 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f7ab007dab0 con 0x7f7abc156050 2024-04-25T01:05:40.076 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7a9c06be30 msgr2=0x7f7a9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009560 tx=0x7f7aac009190 comp rx=0 tx=0).stop 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 msgr2=0x7f7abc1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 0x7f7abc1581f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0000c00 tx=0x7f7ab00046c0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 shutdown_connections 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7abc159630 0x7f7abc1eb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7abc156050 0x7f7abc1581f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 --2- 172.21.15.129:0/3246414561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7abc0c8cc0 0x7f7abc0c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 >> 172.21.15.129:0/3246414561 conn(0x7f7abc06c400 msgr2=0x7f7abc0741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 shutdown_connections 2024-04-25T01:05:40.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.083+0000 7f7ac0e1c640 1 -- 172.21.15.129:0/3246414561 wait complete. 2024-04-25T01:05:40.091 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:05:40.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.200+0000 7f9fa8e6d640 1 Processor -- start 2024-04-25T01:05:40.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.200+0000 7f9fa8e6d640 1 -- start start 2024-04-25T01:05:40.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa8e6d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa40c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa8e6d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9fa406e540 0x7f9fa40c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa8e6d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9fa40a7d00 0x7f9fa41569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa8e6d640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa40c4f40 con 0x7f9fa406e540 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa8e6d640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa4157730 con 0x7f9fa40a7d00 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa8e6d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa4070460 con 0x7f9fa41508d0 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9fa406e540 0x7f9fa40c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f9fa406e540 0x7f9fa40c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55528/0 (socket says 172.21.15.129:55528) 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 -- 172.21.15.129:0/4292059028 learned_addr learned my addr 172.21.15.129:0/4292059028 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:40.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa259c640 1 --2- 172.21.15.129:0/4292059028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa40c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa2d9d640 1 --2- 172.21.15.129:0/4292059028 >> v2:172.21.15.171:3300/0 conn(0x7f9fa40a7d00 0x7f9fa41569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 -- 172.21.15.129:0/4292059028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 msgr2=0x7f9fa40c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 --2- 172.21.15.129:0/4292059028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa40c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 -- 172.21.15.129:0/4292059028 >> v2:172.21.15.171:3300/0 conn(0x7f9fa40a7d00 msgr2=0x7f9fa41569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 --2- 172.21.15.129:0/4292059028 >> v2:172.21.15.171:3300/0 conn(0x7f9fa40a7d00 0x7f9fa41569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa1d9b640 1 -- 172.21.15.129:0/4292059028 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa4070740 con 0x7f9fa406e540 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.201+0000 7f9fa2d9d640 1 --2- 172.21.15.129:0/4292059028 >> v2:172.21.15.171:3300/0 conn(0x7f9fa40a7d00 0x7f9fa41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.202+0000 7f9fa1d9b640 1 --2- 172.21.15.129:0/4292059028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa40c4a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ab720 tx=0x7f9f9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ec79953a935d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.202+0000 7f9fa159a640 1 -- 172.21.15.129:0/4292059028 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98037070 con 0x7f9fa406e540 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.202+0000 7f9fa159a640 1 -- 172.21.15.129:0/4292059028 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f98032440 con 0x7f9fa406e540 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.202+0000 7f9fa159a640 1 -- 172.21.15.129:0/4292059028 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98032810 con 0x7f9fa406e540 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.202+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/4292059028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 msgr2=0x7f9fa40c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.202+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/4292059028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa40c4a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ab720 tx=0x7f9f9802a1e0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/4292059028 shutdown_connections 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/4292059028 >> v2:172.21.15.171:3300/0 conn(0x7f9fa40a7d00 0x7f9fa41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/4292059028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa40c4a00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/4292059028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa40c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/4292059028 >> 172.21.15.129:0/4292059028 conn(0x7f9fa40aa740 msgr2=0x7f9fa40aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/4292059028 shutdown_connections 2024-04-25T01:05:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.203+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/4292059028 wait complete. 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.204+0000 7f9fa8e6d640 1 Processor -- start 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.204+0000 7f9fa8e6d640 1 -- start start 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa4149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fa40a7d00 0x7f9fa4151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4150e30 con 0x7f9fa406e540 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4150cb0 con 0x7f9fa40a7d00 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa4156f10 con 0x7f9fa41508d0 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa4149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa4149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55538/0 (socket says 172.21.15.129:55538) 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 -- 172.21.15.129:0/854615407 learned_addr learned my addr 172.21.15.129:0/854615407 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa1d9b640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fa40a7d00 0x7f9fa4151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa2d9d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 -- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 msgr2=0x7f9fa4153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa4153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 -- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fa40a7d00 msgr2=0x7f9fa4151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fa40a7d00 0x7f9fa4151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 -- 172.21.15.129:0/854615407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa4152ce0 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa259c640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa4149d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e870 tx=0x7f9f9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94017070 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9400c8e0 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa41e9a40 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400ccd0 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.205+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa4153280 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.206+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa4075550 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.207+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f9401a070 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.207+0000 7f9f92ffd640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9f8406be30 0x7f9f8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.207+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9f9408b310 con 0x7f9fa406e540 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.207+0000 7f9fa1d9b640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9f8406be30 0x7f9f8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.208+0000 7f9fa1d9b640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9f8406be30 0x7f9f8406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9f98034000 tx=0x7f9f9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.212+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f94059300 con 0x7f9fa406e540 2024-04-25T01:05:40.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.404+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9fa40c5ff0 con 0x7f9fa406e540 2024-04-25T01:05:40.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.404+0000 7f9f92ffd640 1 -- 172.21.15.129:0/854615407 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f9f9405d600 con 0x7f9fa406e540 2024-04-25T01:05:40.400 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9f8406be30 msgr2=0x7f9f8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9f8406be30 0x7f9f8406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9f98034000 tx=0x7f9f9802e000 comp rx=0 tx=0).stop 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 msgr2=0x7f9fa4149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa4149d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e870 tx=0x7f9f9400ed30 comp rx=0 tx=0).stop 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 shutdown_connections 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9f8406be30 0x7f9f8406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.402 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9fa41508d0 0x7f9fa4153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fa40a7d00 0x7f9fa4151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 --2- 172.21.15.129:0/854615407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9fa406e540 0x7f9fa4149d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 >> 172.21.15.129:0/854615407 conn(0x7f9fa40aa740 msgr2=0x7f9fa40a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 shutdown_connections 2024-04-25T01:05:40.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.406+0000 7f9fa8e6d640 1 -- 172.21.15.129:0/854615407 wait complete. 2024-04-25T01:05:40.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:05:40.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.525+0000 7fa6a9112640 1 Processor -- start 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.525+0000 7fa6a9112640 1 -- start start 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a9112640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa6a40ac600 0x7fa6a408b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a9112640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a406e540 0x7fa6a406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a9112640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a40c64b0 con 0x7fa6a406dc00 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a9112640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a40c4f60 con 0x7fa6a40ac600 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a9112640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a40c5240 con 0x7fa6a406e540 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a2d9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa6a40ac600 0x7fa6a408b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a2d9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fa6a40ac600 0x7fa6a408b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55552/0 (socket says 172.21.15.129:55552) 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a359e640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fa6a406dc00 0x7fa6a406c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55428/0 (socket says 172.21.15.129:55428) 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a2d9d640 1 -- 172.21.15.129:0/3471502441 learned_addr learned my addr 172.21.15.129:0/3471502441 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a259c640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a406e540 0x7fa6a406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.526+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1160919436 0 0) 0x7fa6a40c64b0 con 0x7fa6a406dc00 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.552+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa68c003520 con 0x7fa6a406dc00 2024-04-25T01:05:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1681365040 0 0) 0x7fa68c003520 con 0x7fa6a406dc00 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a406e540 msgr2=0x7fa6a406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a406e540 0x7fa6a406d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 >> v2:172.21.15.129:3300/0 conn(0x7fa6a40ac600 msgr2=0x7fa6a408b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 --2- 172.21.15.129:0/3471502441 >> v2:172.21.15.129:3300/0 conn(0x7fa6a40ac600 0x7fa6a408b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a40c5520 con 0x7fa6a406dc00 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7fa69c0019f0 con 0x7fa6a406dc00 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 >> v1:172.21.15.129:6789/0 conn(0x7fa6a406dc00 legacy=0x7fa6a406c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa68c0039b0 0x7fa68c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa68c0043a0 0x7fa68c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c004f90 0x7fa68c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40c4f60 con 0x7fa68c0043a0 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40c5240 con 0x7fa68c004f90 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa68c00bf30 con 0x7fa68c0039b0 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a2d9d640 1 --2- 172.21.15.129:0/3471502441 >> v2:172.21.15.129:3300/0 conn(0x7fa6a40ac600 0x7fa6a408b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a259c640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa68c0043a0 0x7fa68c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a359e640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c004f90 0x7fa68c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a2d9d640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa68c0039b0 0x7fa68c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a259c640 1 -- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa68c0039b0 msgr2=0x7fa68c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a259c640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa68c0039b0 0x7fa68c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a259c640 1 -- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c004f90 msgr2=0x7fa68c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a259c640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c004f90 0x7fa68c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.553+0000 7fa6a259c640 1 -- 172.21.15.129:0/3471502441 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c003520 con 0x7fa68c0043a0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.554+0000 7fa6a259c640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa68c0043a0 0x7fa68c004880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40c3270 tx=0x7fa68c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2561d073dbf20587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.554+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69803a070 con 0x7fa68c0043a0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.554+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa698005d30 con 0x7fa68c0043a0 2024-04-25T01:05:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.554+0000 7fa6a1d9b640 1 -- 172.21.15.129:0/3471502441 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69800de00 con 0x7fa68c0043a0 2024-04-25T01:05:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a9112640 1 -- 172.21.15.129:0/3471502441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa68c0043a0 msgr2=0x7fa68c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a9112640 1 --2- 172.21.15.129:0/3471502441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa68c0043a0 0x7fa68c004880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6a40c3270 tx=0x7fa68c005830 comp rx=0 tx=0).stop 2024-04-25T01:05:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a359e640 1 -- 172.21.15.129:0/3471502441 reap_dead start 2024-04-25T01:05:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a9112640 1 -- 172.21.15.129:0/3471502441 shutdown_connections 2024-04-25T01:05:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a9112640 1 -- 172.21.15.129:0/3471502441 >> 172.21.15.129:0/3471502441 conn(0x7fa6a40a86b0 msgr2=0x7fa6a40a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a9112640 1 -- 172.21.15.129:0/3471502441 shutdown_connections 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.555+0000 7fa6a9112640 1 -- 172.21.15.129:0/3471502441 wait complete. 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.556+0000 7fa6a9112640 1 Processor -- start 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.556+0000 7fa6a9112640 1 -- start start 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a9112640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c0043a0 0x7fa6a41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a9112640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a40a5e70 0x7fa6a408a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a9112640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 0x7fa6a41fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a9112640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a406d6c0 con 0x7fa6a408ab20 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a9112640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40c6140 con 0x7fa68c0043a0 2024-04-25T01:05:40.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a9112640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a406d840 con 0x7fa6a40a5e70 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 0x7fa6a41fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a2d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c0043a0 0x7fa6a41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 0x7fa6a41fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55574/0 (socket says 172.21.15.129:55574) 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a359e640 1 -- 172.21.15.129:0/356124437 learned_addr learned my addr 172.21.15.129:0/356124437 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.557+0000 7fa6a259c640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a40a5e70 0x7fa6a408a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a359e640 1 -- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a40a5e70 msgr2=0x7fa6a408a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a359e640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a40a5e70 0x7fa6a408a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a359e640 1 -- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c0043a0 msgr2=0x7fa6a41597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a359e640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c0043a0 0x7fa6a41597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a359e640 1 -- 172.21.15.129:0/356124437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a4158770 con 0x7fa6a408ab20 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a359e640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 0x7fa6a41fb6b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007600 tx=0x7fa69c007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c018070 con 0x7fa6a408ab20 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c00fa40 con 0x7fa6a408ab20 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a415aba0 con 0x7fa6a408ab20 2024-04-25T01:05:40.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00fdf0 con 0x7fa6a408ab20 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.558+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a415b160 con 0x7fa6a408ab20 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.559+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a4075550 con 0x7fa6a408ab20 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.559+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa69c01b070 con 0x7fa6a408ab20 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.560+0000 7fa68b7fe640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa69006be60 0x7fa69006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.560+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa69c08a830 con 0x7fa6a408ab20 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.560+0000 7fa6a2d9d640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa69006be60 0x7fa69006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.560+0000 7fa6a2d9d640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa69006be60 0x7fa69006e320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa694008520 tx=0x7fa69400a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.564+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa69c058790 con 0x7fa6a408ab20 2024-04-25T01:05:40.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.746+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6a40c4820 con 0x7fa6a408ab20 2024-04-25T01:05:40.742 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.747+0000 7fa68b7fe640 1 -- 172.21.15.129:0/356124437 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa69c05ca90 con 0x7fa6a408ab20 2024-04-25T01:05:40.742 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:05:40.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa69006be60 msgr2=0x7fa69006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa69006be60 0x7fa69006e320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa694008520 tx=0x7fa69400a040 comp rx=0 tx=0).stop 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 msgr2=0x7fa6a41fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 0x7fa6a41fb6b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007600 tx=0x7fa69c007ac0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 shutdown_connections 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fa69006be60 0x7fa69006e320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa6a40a5e70 0x7fa6a408a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6a408ab20 0x7fa6a41fb6b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 --2- 172.21.15.129:0/356124437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa68c0043a0 0x7fa6a41597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 >> 172.21.15.129:0/356124437 conn(0x7fa6a40a86b0 msgr2=0x7fa6a40a7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 shutdown_connections 2024-04-25T01:05:40.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.749+0000 7fa6a9112640 1 -- 172.21.15.129:0/356124437 wait complete. 2024-04-25T01:05:40.755 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:05:40.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 366120 15.51 MB/sec execute 21 sec latency 44.492 ms 2024-04-25T01:05:40.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.871+0000 7fe5e8942640 1 Processor -- start 2024-04-25T01:05:40.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.871+0000 7fe5e8942640 1 -- start start 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e8942640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 0x7fe5e4159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e8942640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe5e40c9f70 0x7fe5e415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e8942640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e40c94e0 con 0x7fe5e40c8cc0 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e8942640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fe5e40c5c10 con 0x7fe5e40c9f70 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e8942640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e40c5650 con 0x7fe5e40c90a0 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e2d9d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe5e40c9f70 0x7fe5e415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5d9d9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 0x7fe5e4159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e2d9d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe5e40c9f70 0x7fe5e415ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:46878/0 (socket says 172.21.15.129:46878) 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5d9d9b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 0x7fe5e4159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:57320/0 (socket says 172.21.15.129:57320) 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e259c640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fe5e40c8cc0 0x7fe5e4156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:46422/0 (socket says 172.21.15.129:46422) 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.872+0000 7fe5e2d9d640 1 -- 172.21.15.129:0/3220719032 learned_addr learned my addr 172.21.15.129:0/3220719032 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.873+0000 7fe5e2d9d640 1 -- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 msgr2=0x7fe5e4159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.873+0000 7fe5e2d9d640 1 --2- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 0x7fe5e4159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.873+0000 7fe5e2d9d640 1 -- 172.21.15.129:0/3220719032 >> v1:172.21.15.171:6789/0 conn(0x7fe5e40c8cc0 legacy=0x7fe5e4156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.873+0000 7fe5e2d9d640 1 -- 172.21.15.129:0/3220719032 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e40c5930 con 0x7fe5e40c9f70 2024-04-25T01:05:40.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.873+0000 7fe5d9d9b640 1 --2- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 0x7fe5e4159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:40.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.873+0000 7fe5e2d9d640 1 --2- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e415ff50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009e50 tx=0x7fe5dc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4dda6d5347440ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e1d9b640 1 -- 172.21.15.129:0/3220719032 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc036070 con 0x7fe5e40c9f70 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e1d9b640 1 -- 172.21.15.129:0/3220719032 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5dc031600 con 0x7fe5e40c9f70 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e1d9b640 1 -- 172.21.15.129:0/3220719032 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0319d0 con 0x7fe5e40c9f70 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 -- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 msgr2=0x7fe5e415ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e415ff50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009e50 tx=0x7fe5dc02a680 comp rx=0 tx=0).stop 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 -- 172.21.15.129:0/3220719032 shutdown_connections 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e415ff50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3220719032 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c90a0 0x7fe5e4159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 -- 172.21.15.129:0/3220719032 >> 172.21.15.129:0/3220719032 conn(0x7fe5e406c400 msgr2=0x7fe5e406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 -- 172.21.15.129:0/3220719032 shutdown_connections 2024-04-25T01:05:40.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.874+0000 7fe5e8942640 1 -- 172.21.15.129:0/3220719032 wait complete. 2024-04-25T01:05:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 Processor -- start 2024-04-25T01:05:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 -- start start 2024-04-25T01:05:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c8cc0 0x7fe5e414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 0x7fe5e4149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e4149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e406d1f0 con 0x7fe5e40c90a0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e40ca400 con 0x7fe5e40c9f70 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.876+0000 7fe5e8942640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e40c5f30 con 0x7fe5e40c8cc0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 0x7fe5e4149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5e259c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c8cc0 0x7fe5e414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5e2d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e4149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 0x7fe5e4149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55582/0 (socket says 172.21.15.129:55582) 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 -- 172.21.15.129:0/3403328395 learned_addr learned my addr 172.21.15.129:0/3403328395 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 -- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c8cc0 msgr2=0x7fe5e414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c8cc0 0x7fe5e414ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 -- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 msgr2=0x7fe5e4149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e4149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 -- 172.21.15.129:0/3403328395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e4155180 con 0x7fe5e40c90a0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5d9d9b640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 0x7fe5e4149680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00e960 tx=0x7fe5cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc017070 con 0x7fe5e40c90a0 2024-04-25T01:05:40.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5cc00ca00 con 0x7fe5e40c90a0 2024-04-25T01:05:40.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e4155460 con 0x7fe5e40c90a0 2024-04-25T01:05:40.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc00cdd0 con 0x7fe5e40c90a0 2024-04-25T01:05:40.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.877+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e414a1d0 con 0x7fe5e40c90a0 2024-04-25T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.878+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e4075550 con 0x7fe5e40c90a0 2024-04-25T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.879+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5cc01a070 con 0x7fe5e40c90a0 2024-04-25T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.879+0000 7fe5db7fe640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe5c406be30 0x7fe5c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.879+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe5cc08b390 con 0x7fe5e40c90a0 2024-04-25T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.879+0000 7fe5e259c640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe5c406be30 0x7fe5c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.880+0000 7fe5e259c640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe5c406be30 0x7fe5c406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4154dd0 tx=0x7fe5d4007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:40.878 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:40.883+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5cc059380 con 0x7fe5e40c90a0 2024-04-25T01:05:41.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.106+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe5e4152f80 con 0x7fe5e40c90a0 2024-04-25T01:05:41.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.106+0000 7fe5db7fe640 1 -- 172.21.15.129:0/3403328395 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fe5cc05d680 con 0x7fe5e40c90a0 2024-04-25T01:05:41.102 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe5c406be30 msgr2=0x7fe5c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe5c406be30 0x7fe5c406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4154dd0 tx=0x7fe5d4007e80 comp rx=0 tx=0).stop 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 msgr2=0x7fe5e4149680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 0x7fe5e4149680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00e960 tx=0x7fe5cc00ee20 comp rx=0 tx=0).stop 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 shutdown_connections 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe5c406be30 0x7fe5c406e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5e40c9f70 0x7fe5e4149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e40c90a0 0x7fe5e4149680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 --2- 172.21.15.129:0/3403328395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe5e40c8cc0 0x7fe5e414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 >> 172.21.15.129:0/3403328395 conn(0x7fe5e406c400 msgr2=0x7fe5e40cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.108+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 shutdown_connections 2024-04-25T01:05:41.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.109+0000 7fe5e8942640 1 -- 172.21.15.129:0/3403328395 wait complete. 2024-04-25T01:05:41.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:05:41.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dccf9d640 1 Processor -- start 2024-04-25T01:05:41.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dccf9d640 1 -- start start 2024-04-25T01:05:41.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dccf9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5dc80c5520 0x7f5dc80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dccf9d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc80c9860 con 0x7f5dc8152fa0 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dccf9d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc80c5a70 con 0x7f5dc81561e0 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dccf9d640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc80c5d50 con 0x7f5dc80c5520 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dbffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5dc80c5520 0x7f5dc80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dc659c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f5dc8152fa0 0x7f5dc80c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55436/0 (socket says 172.21.15.129:55436) 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dbffff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5dc80c5520 0x7f5dc80ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55596/0 (socket says 172.21.15.129:55596) 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.223+0000 7f5dc659c640 1 -- 172.21.15.129:0/2863269819 learned_addr learned my addr 172.21.15.129:0/2863269819 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.224+0000 7f5dbffff640 1 -- 172.21.15.129:0/2863269819 >> v1:172.21.15.171:6789/0 conn(0x7f5dc81561e0 legacy=0x7f5dc80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.224+0000 7f5dbffff640 1 -- 172.21.15.129:0/2863269819 >> v1:172.21.15.129:6789/0 conn(0x7f5dc8152fa0 legacy=0x7f5dc80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.224+0000 7f5dbffff640 1 -- 172.21.15.129:0/2863269819 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc80c7650 con 0x7f5dc80c5520 2024-04-25T01:05:41.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.224+0000 7f5dc5d9b640 1 -- 172.21.15.129:0/2863269819 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765950864 0 0) 0x7f5db8001110 con 0x7f5dc8152fa0 2024-04-25T01:05:41.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.224+0000 7f5dbffff640 1 --2- 172.21.15.129:0/2863269819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc80c5520 0x7f5dc80ca2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5db00097a0 tx=0x7f5db002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dae396a3bbe7862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.224+0000 7f5dc5d9b640 1 -- 172.21.15.129:0/2863269819 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0037070 con 0x7f5dc80c5520 2024-04-25T01:05:41.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dc5d9b640 1 -- 172.21.15.129:0/2863269819 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db0032440 con 0x7f5dc80c5520 2024-04-25T01:05:41.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dc5d9b640 1 -- 172.21.15.129:0/2863269819 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db002ae60 con 0x7f5dc80c5520 2024-04-25T01:05:41.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 -- 172.21.15.129:0/2863269819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc80c5520 msgr2=0x7f5dc80ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/2863269819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc80c5520 0x7f5dc80ca2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5db00097a0 tx=0x7f5db002a110 comp rx=0 tx=0).stop 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 -- 172.21.15.129:0/2863269819 shutdown_connections 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/2863269819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc80c5520 0x7f5dc80ca2e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 -- 172.21.15.129:0/2863269819 >> 172.21.15.129:0/2863269819 conn(0x7f5dc806c400 msgr2=0x7f5dc806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 -- 172.21.15.129:0/2863269819 shutdown_connections 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.225+0000 7f5dccf9d640 1 -- 172.21.15.129:0/2863269819 wait complete. 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.226+0000 7f5dccf9d640 1 Processor -- start 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.226+0000 7f5dccf9d640 1 -- start start 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dccf9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc80c5520 0x7f5dc8158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dccf9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 0x7f5dc80c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dccf9d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dc81561e0 0x7f5dc80bf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dccf9d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80c93d0 con 0x7f5dc8152fa0 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dccf9d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80c6030 con 0x7f5dc80c5520 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dccf9d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc80c9550 con 0x7f5dc81561e0 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 0x7f5dc80c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 0x7f5dc80c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55610/0 (socket says 172.21.15.129:55610) 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 -- 172.21.15.129:0/1825347156 learned_addr learned my addr 172.21.15.129:0/1825347156 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dc659c640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc80c5520 0x7f5dc8158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dc6d9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dc81561e0 0x7f5dc80bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 -- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dc81561e0 msgr2=0x7f5dc80bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dc81561e0 0x7f5dc80bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 -- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc80c5520 msgr2=0x7f5dc8158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.227+0000 7f5dbffff640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc80c5520 0x7f5dc8158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dbffff640 1 -- 172.21.15.129:0/1825347156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc80c39b0 con 0x7f5dc8152fa0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dbffff640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 0x7f5dc80c3060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0037040 con 0x7f5dc8152fa0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db0042070 con 0x7f5dc8152fa0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc81581a0 con 0x7f5dc8152fa0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0030a70 con 0x7f5dc8152fa0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.228+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc8158760 con 0x7f5dc8152fa0 2024-04-25T01:05:41.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.229+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc806d480 con 0x7f5dc8152fa0 2024-04-25T01:05:41.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.229+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5db0030c10 con 0x7f5dc8152fa0 2024-04-25T01:05:41.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.230+0000 7f5dbeffd640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5da806be60 0x7f5da806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.230+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5db00abb30 con 0x7f5dc8152fa0 2024-04-25T01:05:41.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.230+0000 7f5dc659c640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5da806be60 0x7f5da806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.230+0000 7f5dc659c640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5da806be60 0x7f5da806e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000f60 tx=0x7f5db8007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.234+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5dc806d480 con 0x7f5dc8152fa0 2024-04-25T01:05:41.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.445+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5dc80c8770 con 0x7f5dc8152fa0 2024-04-25T01:05:41.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.445+0000 7f5dbeffd640 1 -- 172.21.15.129:0/1825347156 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5dc80c8770 con 0x7f5dc8152fa0 2024-04-25T01:05:41.440 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5da806be60 msgr2=0x7f5da806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5da806be60 0x7f5da806e320 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000f60 tx=0x7f5db8007b50 comp rx=0 tx=0).stop 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 msgr2=0x7f5dc80c3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 0x7f5dc80c3060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db00046c0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 shutdown_connections 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f5da806be60 0x7f5da806e320 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dc81561e0 0x7f5dc80bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dc8152fa0 0x7f5dc80c3060 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 --2- 172.21.15.129:0/1825347156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dc80c5520 0x7f5dc8158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 >> 172.21.15.129:0/1825347156 conn(0x7f5dc806c400 msgr2=0x7f5dc80abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 shutdown_connections 2024-04-25T01:05:41.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.447+0000 7f5dccf9d640 1 -- 172.21.15.129:0/1825347156 wait complete. 2024-04-25T01:05:41.454 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-25T01:05:41.454 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:05:41.470 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:05:41.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 Processor -- start 2024-04-25T01:05:41.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 -- start start 2024-04-25T01:05:41.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fca54152fa0 0x7fca540ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fca541561e0 0x7fca540c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca540c9860 con 0x7fca540c5520 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fca540c5a70 con 0x7fca54152fa0 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.615+0000 7fca59edf640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fca540c5d50 con 0x7fca541561e0 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fca54152fa0 0x7fca540ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fca54152fa0 0x7fca540ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55626/0 (socket says 172.21.15.129:55626) 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 -- 172.21.15.129:0/3765845911 learned_addr learned my addr 172.21.15.129:0/3765845911 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca53fff640 1 --2- 172.21.15.129:0/3765845911 >> v2:172.21.15.171:3300/0 conn(0x7fca541561e0 0x7fca540c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 -- 172.21.15.129:0/3765845911 >> v2:172.21.15.171:3300/0 conn(0x7fca541561e0 msgr2=0x7fca540c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 --2- 172.21.15.129:0/3765845911 >> v2:172.21.15.171:3300/0 conn(0x7fca541561e0 0x7fca540c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 -- 172.21.15.129:0/3765845911 >> v1:172.21.15.171:6789/0 conn(0x7fca540c5520 legacy=0x7fca540c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 -- 172.21.15.129:0/3765845911 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca540c7650 con 0x7fca54152fa0 2024-04-25T01:05:41.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.616+0000 7fca537fe640 1 --2- 172.21.15.129:0/3765845911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca540ca2e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca540aa580 tx=0x7fca4402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40366b6de8b55e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca527fc640 1 -- 172.21.15.129:0/3765845911 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44037070 con 0x7fca54152fa0 2024-04-25T01:05:41.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca527fc640 1 -- 172.21.15.129:0/3765845911 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca44032440 con 0x7fca54152fa0 2024-04-25T01:05:41.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca527fc640 1 -- 172.21.15.129:0/3765845911 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44032810 con 0x7fca54152fa0 2024-04-25T01:05:41.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 -- 172.21.15.129:0/3765845911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 msgr2=0x7fca540ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 --2- 172.21.15.129:0/3765845911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca540ca2e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca540aa580 tx=0x7fca4402a150 comp rx=0 tx=0).stop 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 -- 172.21.15.129:0/3765845911 shutdown_connections 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 --2- 172.21.15.129:0/3765845911 >> v2:172.21.15.171:3300/0 conn(0x7fca541561e0 0x7fca540c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 --2- 172.21.15.129:0/3765845911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca540ca2e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 -- 172.21.15.129:0/3765845911 >> 172.21.15.129:0/3765845911 conn(0x7fca5406c400 msgr2=0x7fca5406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 -- 172.21.15.129:0/3765845911 shutdown_connections 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.617+0000 7fca59edf640 1 -- 172.21.15.129:0/3765845911 wait complete. 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.618+0000 7fca59edf640 1 Processor -- start 2024-04-25T01:05:41.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.618+0000 7fca59edf640 1 -- start start 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca59edf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 0x7fca540aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca59edf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca5414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca59edf640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fca541561e0 0x7fca5414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca59edf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fca540c9200 con 0x7fca54152fa0 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca59edf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca540c6030 con 0x7fca540c5520 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca59edf640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fca540c9380 con 0x7fca541561e0 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca52ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca5414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca537fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 0x7fca540aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca537fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 0x7fca540aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:46914/0 (socket says 172.21.15.129:46914) 2024-04-25T01:05:41.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca52ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca5414b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55632/0 (socket says 172.21.15.129:55632) 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca537fe640 1 -- 172.21.15.129:0/1875098640 learned_addr learned my addr 172.21.15.129:0/1875098640 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.619+0000 7fca53fff640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fca541561e0 0x7fca5414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca52ffd640 1 -- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fca541561e0 msgr2=0x7fca5414beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca52ffd640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fca541561e0 0x7fca5414beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca52ffd640 1 -- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 msgr2=0x7fca540aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca52ffd640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 0x7fca540aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca52ffd640 1 -- 172.21.15.129:0/1875098640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca541553e0 con 0x7fca54152fa0 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca537fe640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 0x7fca540aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca52ffd640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca5414b970 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca4800c3f0 tx=0x7fca4800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca480052d0 con 0x7fca54152fa0 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca48005490 con 0x7fca54152fa0 2024-04-25T01:05:41.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48007260 con 0x7fca54152fa0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca541ec6d0 con 0x7fca54152fa0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.620+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca54158650 con 0x7fca54152fa0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.621+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5406b8a0 con 0x7fca54152fa0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.621+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca48007400 con 0x7fca54152fa0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.622+0000 7fca3bfff640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fca4006be30 0x7fca4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.622+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fca4808a4d0 con 0x7fca54152fa0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.622+0000 7fca537fe640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fca4006be30 0x7fca4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.623+0000 7fca537fe640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fca4006be30 0x7fca4006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca44035040 tx=0x7fca4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.626+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fca5406b8a0 con 0x7fca54152fa0 2024-04-25T01:05:41.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.817+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca54075550 con 0x7fca54152fa0 2024-04-25T01:05:41.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.818+0000 7fca3bfff640 1 -- 172.21.15.129:0/1875098640 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18891 (secure 0 0 0) 0x7fca540c9860 con 0x7fca54152fa0 2024-04-25T01:05:41.814 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:05:41.814 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:02:13.734125+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fca4006be30 msgr2=0x7fca4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fca4006be30 0x7fca4006e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca44035040 tx=0x7fca4402e000 comp rx=0 tx=0).stop 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 msgr2=0x7fca5414b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca5414b970 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca4800c3f0 tx=0x7fca4800c8b0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 shutdown_connections 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fca4006be30 0x7fca4006e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fca541561e0 0x7fca5414beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54152fa0 0x7fca5414b970 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 --2- 172.21.15.129:0/1875098640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca540c5520 0x7fca540aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 >> 172.21.15.129:0/1875098640 conn(0x7fca5406c400 msgr2=0x7fca5406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 shutdown_connections 2024-04-25T01:05:41.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.820+0000 7fca59edf640 1 -- 172.21.15.129:0/1875098640 wait complete. 2024-04-25T01:05:41.827 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:05:41.828 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:05:41.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 369550 15.66 MB/sec execute 22 sec latency 31.982 ms 2024-04-25T01:05:41.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.945+0000 7fbf48910640 1 Processor -- start 2024-04-25T01:05:41.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.945+0000 7fbf48910640 1 -- start start 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf48910640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fbf440c90a0 0x7fbf44159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf48910640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbf440c9f70 0x7fbf4415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf48910640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf440c5c10 con 0x7fbf440c8cc0 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf48910640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fbf440c9510 con 0x7fbf440c9f70 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf48910640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fbf440c5650 con 0x7fbf440c90a0 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbf440c9f70 0x7fbf4415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fbf440c9f70 0x7fbf4415ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55648/0 (socket says 172.21.15.129:55648) 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 -- 172.21.15.129:0/2589050703 learned_addr learned my addr 172.21.15.129:0/2589050703 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf41d9b640 1 --2- 172.21.15.129:0/2589050703 >> v2:172.21.15.171:3300/0 conn(0x7fbf440c90a0 0x7fbf44159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 -- 172.21.15.129:0/2589050703 >> v2:172.21.15.171:3300/0 conn(0x7fbf440c90a0 msgr2=0x7fbf44159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:41.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 --2- 172.21.15.129:0/2589050703 >> v2:172.21.15.171:3300/0 conn(0x7fbf440c90a0 0x7fbf44159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 -- 172.21.15.129:0/2589050703 >> v1:172.21.15.171:6789/0 conn(0x7fbf440c8cc0 legacy=0x7fbf44156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.946+0000 7fbf42d9d640 1 -- 172.21.15.129:0/2589050703 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf440c5930 con 0x7fbf440c9f70 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.947+0000 7fbf42d9d640 1 --2- 172.21.15.129:0/2589050703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c9f70 0x7fbf4415ff50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbf440be400 tx=0x7fbf3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8488f4d764e31ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.947+0000 7fbf4159a640 1 -- 172.21.15.129:0/2589050703 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c036070 con 0x7fbf440c9f70 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.947+0000 7fbf4159a640 1 -- 172.21.15.129:0/2589050703 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c031600 con 0x7fbf440c9f70 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.947+0000 7fbf4159a640 1 -- 172.21.15.129:0/2589050703 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0319d0 con 0x7fbf440c9f70 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 -- 172.21.15.129:0/2589050703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c9f70 msgr2=0x7fbf4415ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 --2- 172.21.15.129:0/2589050703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c9f70 0x7fbf4415ff50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbf440be400 tx=0x7fbf3c02a6a0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 -- 172.21.15.129:0/2589050703 shutdown_connections 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 --2- 172.21.15.129:0/2589050703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c9f70 0x7fbf4415ff50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 --2- 172.21.15.129:0/2589050703 >> v2:172.21.15.171:3300/0 conn(0x7fbf440c90a0 0x7fbf44159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 -- 172.21.15.129:0/2589050703 >> 172.21.15.129:0/2589050703 conn(0x7fbf4406c400 msgr2=0x7fbf4406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:41.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 -- 172.21.15.129:0/2589050703 shutdown_connections 2024-04-25T01:05:41.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.948+0000 7fbf48910640 1 -- 172.21.15.129:0/2589050703 wait complete. 2024-04-25T01:05:41.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.949+0000 7fbf48910640 1 Processor -- start 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.949+0000 7fbf48910640 1 -- start start 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf48910640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf440c8cc0 0x7fbf441ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf48910640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 0x7fbf441ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf48910640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf440c9f70 0x7fbf44157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf48910640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4406d1f0 con 0x7fbf440c90a0 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf48910640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440ca3e0 con 0x7fbf440c9f70 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf48910640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fbf440c5f30 con 0x7fbf440c8cc0 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf41d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 0x7fbf441ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf41d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 0x7fbf441ec9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55662/0 (socket says 172.21.15.129:55662) 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf41d9b640 1 -- 172.21.15.129:0/2856915672 learned_addr learned my addr 172.21.15.129:0/2856915672 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf42d9d640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf440c9f70 0x7fbf44157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.950+0000 7fbf4259c640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf440c8cc0 0x7fbf441ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf41d9b640 1 -- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf440c8cc0 msgr2=0x7fbf441ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf41d9b640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf440c8cc0 0x7fbf441ebfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf41d9b640 1 -- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf440c9f70 msgr2=0x7fbf44157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf41d9b640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf440c9f70 0x7fbf44157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf41d9b640 1 -- 172.21.15.129:0/2856915672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf44156bc0 con 0x7fbf440c90a0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf42d9d640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf440c9f70 0x7fbf44157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf41d9b640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 0x7fbf441ec9b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800e870 tx=0x7fbf3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38017070 con 0x7fbf440c90a0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3800c940 con 0x7fbf440c90a0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800cd30 con 0x7fbf440c90a0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf44156ea0 con 0x7fbf440c90a0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.951+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf44157460 con 0x7fbf440c90a0 2024-04-25T01:05:41.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.952+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4406b8a0 con 0x7fbf440c90a0 2024-04-25T01:05:41.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.953+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf3801a070 con 0x7fbf440c90a0 2024-04-25T01:05:41.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.953+0000 7fbf32ffd640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbf2406be30 0x7fbf2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:41.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.954+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbf3808b050 con 0x7fbf440c90a0 2024-04-25T01:05:41.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.954+0000 7fbf4259c640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbf2406be30 0x7fbf2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:41.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.954+0000 7fbf4259c640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbf2406be30 0x7fbf2406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbf440cb640 tx=0x7fbf34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:41.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:41.959+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbf4406b8a0 con 0x7fbf440c90a0 2024-04-25T01:05:42.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.152+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf440c9b40 con 0x7fbf440c90a0 2024-04-25T01:05:42.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.152+0000 7fbf32ffd640 1 -- 172.21.15.129:0/2856915672 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fbf38058fc0 con 0x7fbf440c90a0 2024-04-25T01:05:42.148 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbf2406be30 msgr2=0x7fbf2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbf2406be30 0x7fbf2406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbf440cb640 tx=0x7fbf34009040 comp rx=0 tx=0).stop 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 msgr2=0x7fbf441ec9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 0x7fbf441ec9b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800e870 tx=0x7fbf3800ed30 comp rx=0 tx=0).stop 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 shutdown_connections 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fbf2406be30 0x7fbf2406e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf440c9f70 0x7fbf44157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf440c90a0 0x7fbf441ec9b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 --2- 172.21.15.129:0/2856915672 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf440c8cc0 0x7fbf441ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 >> 172.21.15.129:0/2856915672 conn(0x7fbf4406c400 msgr2=0x7fbf440aa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:42.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 shutdown_connections 2024-04-25T01:05:42.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.154+0000 7fbf48910640 1 -- 172.21.15.129:0/2856915672 wait complete. 2024-04-25T01:05:42.161 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:05:42.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.272+0000 7f3ca4493640 1 Processor -- start 2024-04-25T01:05:42.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.272+0000 7f3ca4493640 1 -- start start 2024-04-25T01:05:42.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.273+0000 7f3ca4493640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 0x7f3c9c159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.273+0000 7f3ca4493640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 0x7f3c9c15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.273+0000 7f3ca4493640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9c0c94e0 con 0x7f3c9c0c8cc0 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.273+0000 7f3ca4493640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f3c9c0c5c10 con 0x7f3c9c0c9f70 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.273+0000 7f3ca4493640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0c5650 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca1a2e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 0x7f3c9c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca2a30640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 0x7f3c9c15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca2a30640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 0x7f3c9c15ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:46938/0 (socket says 172.21.15.129:46938) 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca222f640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f3c9c0c8cc0 0x7f3c9c156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:46458/0 (socket says 172.21.15.129:46458) 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca1a2e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 0x7f3c9c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:57376/0 (socket says 172.21.15.129:57376) 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca222f640 1 -- 172.21.15.129:0/2709901674 learned_addr learned my addr 172.21.15.129:0/2709901674 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/2709901674 >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 msgr2=0x7f3c9c15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:42.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca1a2e640 1 --2- 172.21.15.129:0/2709901674 >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 0x7f3c9c15ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/2709901674 >> v1:172.21.15.171:6789/0 conn(0x7f3c9c0c8cc0 legacy=0x7f3c9c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca122d640 1 -- 172.21.15.129:0/2709901674 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171279106 0 0) 0x7f3c9c0c94e0 con 0x7f3c9c0c8cc0 2024-04-25T01:05:42.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/2709901674 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c0c5930 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.274+0000 7f3ca2a30640 1 --2- 172.21.15.129:0/2709901674 >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 0x7f3c9c15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.275+0000 7f3ca1a2e640 1 --2- 172.21.15.129:0/2709901674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 0x7f3c9c159630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0097a0 tx=0x7f3c8c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c416437c13115d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.275+0000 7f3ca122d640 1 -- 172.21.15.129:0/2709901674 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c037070 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.275+0000 7f3ca122d640 1 -- 172.21.15.129:0/2709901674 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c02ae50 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.275+0000 7f3ca122d640 1 -- 172.21.15.129:0/2709901674 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c032660 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.275+0000 7f3ca4493640 1 -- 172.21.15.129:0/2709901674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 msgr2=0x7f3c9c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.275+0000 7f3ca4493640 1 --2- 172.21.15.129:0/2709901674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 0x7f3c9c159630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0097a0 tx=0x7f3c8c02a100 comp rx=0 tx=0).stop 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.276+0000 7f3ca4493640 1 -- 172.21.15.129:0/2709901674 shutdown_connections 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.276+0000 7f3ca4493640 1 --2- 172.21.15.129:0/2709901674 >> v2:172.21.15.171:3300/0 conn(0x7f3c9c0c9f70 0x7f3c9c15ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.276+0000 7f3ca4493640 1 --2- 172.21.15.129:0/2709901674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c90a0 0x7f3c9c159630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.276+0000 7f3ca4493640 1 -- 172.21.15.129:0/2709901674 >> 172.21.15.129:0/2709901674 conn(0x7f3c9c06c400 msgr2=0x7f3c9c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.276+0000 7f3ca4493640 1 -- 172.21.15.129:0/2709901674 shutdown_connections 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.276+0000 7f3ca4493640 1 -- 172.21.15.129:0/2709901674 wait complete. 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 Processor -- start 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 -- start start 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c9c0c8cc0 0x7f3c9c1e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 0x7f3c9c156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c9f70 0x7f3c9c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c06d1f0 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c0ca400 con 0x7f3c9c0c8cc0 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca4493640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c0c5f30 con 0x7f3c9c0c9f70 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca1a2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 0x7f3c9c156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca222f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c9c0c8cc0 0x7f3c9c1e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca1a2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 0x7f3c9c156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55666/0 (socket says 172.21.15.129:55666) 2024-04-25T01:05:42.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/4270097840 learned_addr learned my addr 172.21.15.129:0/4270097840 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.277+0000 7f3ca2a30640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c9f70 0x7f3c9c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c9f70 msgr2=0x7f3c9c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca1a2e640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c9f70 0x7f3c9c1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c9c0c8cc0 msgr2=0x7f3c9c1e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca1a2e640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c9c0c8cc0 0x7f3c9c1e15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca1a2e640 1 -- 172.21.15.129:0/4270097840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c1df8f0 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca222f640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c9c0c8cc0 0x7f3c9c1e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca1a2e640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 0x7f3c9c156c90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c037040 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c004030 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c1e1af0 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c006e10 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.278+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1dfe40 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.279+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9c075550 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.279+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c8c003610 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.280+0000 7f3c92ffd640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c8006be60 0x7f3c8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.280+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3c8c0abd10 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.280+0000 7f3ca222f640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c8006be60 0x7f3c8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.280+0000 7f3ca222f640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c8006be60 0x7f3c8006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c98007fd0 tx=0x7f3c98007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.284+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c9c1e1af0 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.479+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c9c0c9760 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.480+0000 7f3c92ffd640 1 -- 172.21.15.129:0/4270097840 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f3c8c07a350 con 0x7f3c9c0c90a0 2024-04-25T01:05:42.475 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:05:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c8006be60 msgr2=0x7f3c8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c8006be60 0x7f3c8006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c98007fd0 tx=0x7f3c98007de0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 msgr2=0x7f3c9c156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 0x7f3c9c156c90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 shutdown_connections 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3c8006be60 0x7f3c8006e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c9c0c9f70 0x7f3c9c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c9c0c90a0 0x7f3c9c156c90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 --2- 172.21.15.129:0/4270097840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c9c0c8cc0 0x7f3c9c1e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 >> 172.21.15.129:0/4270097840 conn(0x7f3c9c06c400 msgr2=0x7f3c9c150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 shutdown_connections 2024-04-25T01:05:42.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.482+0000 7f3ca4493640 1 -- 172.21.15.129:0/4270097840 wait complete. 2024-04-25T01:05:42.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:05:42.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 Processor -- start 2024-04-25T01:05:42.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 -- start start 2024-04-25T01:05:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f288c0c90a0 0x7f288c159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f288c0c9f70 0x7f288c15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0c5c10 con 0x7f288c0c8cc0 2024-04-25T01:05:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f288c0c9510 con 0x7f288c0c9f70 2024-04-25T01:05:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.598+0000 7f2891e2c640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f288c0c5650 con 0x7f288c0c90a0 2024-04-25T01:05:42.596 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288bfff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f288c0c9f70 0x7f288c15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288b7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f288c0c8cc0 0x7f288c156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55450/0 (socket says 172.21.15.129:55450) 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288b7fe640 1 -- 172.21.15.129:0/1246138062 learned_addr learned my addr 172.21.15.129:0/1246138062 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288affd640 1 --2- 172.21.15.129:0/1246138062 >> v2:172.21.15.171:3300/0 conn(0x7f288c0c90a0 0x7f288c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288bfff640 1 -- 172.21.15.129:0/1246138062 >> v2:172.21.15.171:3300/0 conn(0x7f288c0c90a0 msgr2=0x7f288c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288bfff640 1 --2- 172.21.15.129:0/1246138062 >> v2:172.21.15.171:3300/0 conn(0x7f288c0c90a0 0x7f288c159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288bfff640 1 -- 172.21.15.129:0/1246138062 >> v1:172.21.15.129:6789/0 conn(0x7f288c0c8cc0 legacy=0x7f288c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288bfff640 1 -- 172.21.15.129:0/1246138062 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c0c5930 con 0x7f288c0c9f70 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.599+0000 7f288a7fc640 1 -- 172.21.15.129:0/1246138062 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167535748 0 0) 0x7f288c0c5c10 con 0x7f288c0c8cc0 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f288bfff640 1 --2- 172.21.15.129:0/1246138062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c15ff50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f288c0be400 tx=0x7f288402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b407496dc4b5641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.597 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f288a7fc640 1 -- 172.21.15.129:0/1246138062 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884007740 con 0x7f288c0c9f70 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f288a7fc640 1 -- 172.21.15.129:0/1246138062 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28840078e0 con 0x7f288c0c9f70 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f288a7fc640 1 -- 172.21.15.129:0/1246138062 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884007cb0 con 0x7f288c0c9f70 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 -- 172.21.15.129:0/1246138062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 msgr2=0x7f288c15ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 --2- 172.21.15.129:0/1246138062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c15ff50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f288c0be400 tx=0x7f288402a5f0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 -- 172.21.15.129:0/1246138062 shutdown_connections 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 --2- 172.21.15.129:0/1246138062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c15ff50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 --2- 172.21.15.129:0/1246138062 >> v2:172.21.15.171:3300/0 conn(0x7f288c0c90a0 0x7f288c159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 -- 172.21.15.129:0/1246138062 >> 172.21.15.129:0/1246138062 conn(0x7f288c06c400 msgr2=0x7f288c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 -- 172.21.15.129:0/1246138062 shutdown_connections 2024-04-25T01:05:42.598 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.600+0000 7f2891e2c640 1 -- 172.21.15.129:0/1246138062 wait complete. 2024-04-25T01:05:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.601+0000 7f2891e2c640 1 Processor -- start 2024-04-25T01:05:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 -- start start 2024-04-25T01:05:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f288c0c8cc0 0x7f288c1ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f288c0c90a0 0x7f288c148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c1ec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f288c06d1f0 con 0x7f288c0c9f70 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f288c0ca3e0 con 0x7f288c0c90a0 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f2891e2c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f288c0c5f30 con 0x7f288c0c8cc0 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f288bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c1ec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f288bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c1ec3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55684/0 (socket says 172.21.15.129:55684) 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f288bfff640 1 -- 172.21.15.129:0/2702672138 learned_addr learned my addr 172.21.15.129:0/2702672138 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f288b7fe640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f288c0c8cc0 0x7f288c1ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.602+0000 7f288affd640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f288c0c90a0 0x7f288c148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f288bfff640 1 -- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f288c0c8cc0 msgr2=0x7f288c1ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f288bfff640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f288c0c8cc0 0x7f288c1ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f288bfff640 1 -- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f288c0c90a0 msgr2=0x7f288c148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f288bfff640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f288c0c90a0 0x7f288c148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f288bfff640 1 -- 172.21.15.129:0/2702672138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c1ef020 con 0x7f288c0c9f70 2024-04-25T01:05:42.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f288bfff640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c1ec3e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f288402abe0 tx=0x7f2884004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288403f070 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2884031bc0 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c1eb080 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884009450 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.603+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c1eb640 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.604+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c075550 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.604+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f288403b410 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.605+0000 7f2873fff640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f287806be30 0x7f287806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.605+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f28840ac800 con 0x7f288c0c9f70 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.605+0000 7f288b7fe640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f287806be30 0x7f287806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.605+0000 7f288b7fe640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f287806be30 0x7f287806e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f287c007fd0 tx=0x7f287c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.609+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f288407a7f0 con 0x7f288c0c9f70 2024-04-25T01:05:42.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.794+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f288c0c1a30 con 0x7f288c0c9f70 2024-04-25T01:05:42.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.794+0000 7f2873fff640 1 -- 172.21.15.129:0/2702672138 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f288407eaf0 con 0x7f288c0c9f70 2024-04-25T01:05:42.789 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f287806be30 msgr2=0x7f287806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f287806be30 0x7f287806e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f287c007fd0 tx=0x7f287c007de0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 msgr2=0x7f288c1ec3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c1ec3e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f288402abe0 tx=0x7f2884004a40 comp rx=0 tx=0).stop 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 shutdown_connections 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f287806be30 0x7f287806e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f288c0c9f70 0x7f288c1ec3e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f288c0c90a0 0x7f288c148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 --2- 172.21.15.129:0/2702672138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f288c0c8cc0 0x7f288c1ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 >> 172.21.15.129:0/2702672138 conn(0x7f288c06c400 msgr2=0x7f288c14b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 shutdown_connections 2024-04-25T01:05:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.796+0000 7f2891e2c640 1 -- 172.21.15.129:0/2702672138 wait complete. 2024-04-25T01:05:42.803 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:05:42.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 372992 15.80 MB/sec execute 23 sec latency 13.745 ms 2024-04-25T01:05:42.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.908+0000 7f9c00e9a640 1 Processor -- start 2024-04-25T01:05:42.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.908+0000 7f9c00e9a640 1 -- start start 2024-04-25T01:05:42.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9c00e9a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9bfc0c5fb0 0x7f9bfc06df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9c00e9a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bfc0c6520 con 0x7f9bfc06e490 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9c00e9a640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bfc151ca0 con 0x7f9bfc0abcd0 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9c00e9a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f9bfc1522f0 con 0x7f9bfc0c5fb0 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bfad9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f9bfc06e490 0x7f9bfc06c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55456/0 (socket says 172.21.15.129:55456) 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bfad9d640 1 -- 172.21.15.129:0/858377695 learned_addr learned my addr 172.21.15.129:0/858377695 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf3fff640 1 --2- 172.21.15.129:0/858377695 >> v2:172.21.15.171:3300/0 conn(0x7f9bfc0c5fb0 0x7f9bfc06df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547633985 0 0) 0x7f9bfc0c6520 con 0x7f9bfc06e490 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be8003520 con 0x7f9bfc06e490 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299863017 0 0) 0x7f9bfc151ca0 con 0x7f9bfc0abcd0 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bfc0c6520 con 0x7f9bfc0abcd0 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf3fff640 1 -- 172.21.15.129:0/858377695 >> v1:172.21.15.171:6789/0 conn(0x7f9bfc0abcd0 legacy=0x7f9bfc0c1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf3fff640 1 -- 172.21.15.129:0/858377695 >> v1:172.21.15.129:6789/0 conn(0x7f9bfc06e490 legacy=0x7f9bfc06c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.909+0000 7f9bf3fff640 1 -- 172.21.15.129:0/858377695 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc151f80 con 0x7f9bfc0c5fb0 2024-04-25T01:05:42.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.910+0000 7f9bf3fff640 1 --2- 172.21.15.129:0/858377695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc06df50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32807ed85a0e2b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.910+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4037070 con 0x7f9bfc0c5fb0 2024-04-25T01:05:42.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.910+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4032440 con 0x7f9bfc0c5fb0 2024-04-25T01:05:42.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.910+0000 7f9bf9d9b640 1 -- 172.21.15.129:0/858377695 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be40327f0 con 0x7f9bfc0c5fb0 2024-04-25T01:05:42.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 -- 172.21.15.129:0/858377695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 msgr2=0x7f9bfc06df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/858377695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc06df50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097a0 tx=0x7f9be402a1a0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 -- 172.21.15.129:0/858377695 shutdown_connections 2024-04-25T01:05:42.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/858377695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc06df50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 -- 172.21.15.129:0/858377695 >> 172.21.15.129:0/858377695 conn(0x7f9bfc0a86b0 msgr2=0x7f9bfc0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 -- 172.21.15.129:0/858377695 shutdown_connections 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.911+0000 7f9c00e9a640 1 -- 172.21.15.129:0/858377695 wait complete. 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.912+0000 7f9c00e9a640 1 Processor -- start 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.912+0000 7f9c00e9a640 1 -- start start 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9c00e9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 0x7f9bfc0bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9c00e9a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9bfc0abcd0 0x7f9bfc1ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9c00e9a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc1de070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9c00e9a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc0ac0b0 con 0x7f9bfc06e490 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9c00e9a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc0c1b30 con 0x7f9bfc0c5fb0 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9c00e9a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc0c0780 con 0x7f9bfc0abcd0 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 0x7f9bfc0bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 0x7f9bfc0bffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55688/0 (socket says 172.21.15.129:55688) 2024-04-25T01:05:42.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 -- 172.21.15.129:0/514218707 learned_addr learned my addr 172.21.15.129:0/514218707 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfad9d640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc1de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bf3fff640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9bfc0abcd0 0x7f9bfc1ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 -- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9bfc0abcd0 msgr2=0x7f9bfc1ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9bfc0abcd0 0x7f9bfc1ddb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 -- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 msgr2=0x7f9bfc1de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc1de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.913+0000 7f9bfa59c640 1 -- 172.21.15.129:0/514218707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc1ea990 con 0x7f9bfc06e490 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.914+0000 7f9bfa59c640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 0x7f9bfc0bffa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c500 tx=0x7f9bec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.914+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0052d0 con 0x7f9bfc06e490 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.914+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc1dd330 con 0x7f9bfc06e490 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.914+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec005490 con 0x7f9bfc06e490 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.914+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec012870 con 0x7f9bfc06e490 2024-04-25T01:05:42.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.914+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc1deca0 con 0x7f9bfc06e490 2024-04-25T01:05:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.915+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bfc075550 con 0x7f9bfc06e490 2024-04-25T01:05:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.915+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bec012a10 con 0x7f9bfc06e490 2024-04-25T01:05:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.916+0000 7f9bf2ffd640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.916+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9bec08ae20 con 0x7f9bfc06e490 2024-04-25T01:05:42.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.916+0000 7f9bf3fff640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:42.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.917+0000 7f9bf3fff640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034000 tx=0x7f9be402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:42.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:42.920+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9bec058d90 con 0x7f9bfc06e490 2024-04-25T01:05:43.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.132+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9bfc0c0fe0 con 0x7f9bfc06e490 2024-04-25T01:05:43.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.132+0000 7f9bf2ffd640 1 -- 172.21.15.129:0/514218707 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9bec05d090 con 0x7f9bfc06e490 2024-04-25T01:05:43.128 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9bdc06be30 msgr2=0x7f9bdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9be4034000 tx=0x7f9be402e000 comp rx=0 tx=0).stop 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 msgr2=0x7f9bfc0bffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 0x7f9bfc0bffa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c500 tx=0x7f9bec00c9c0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 shutdown_connections 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f9bdc06be30 0x7f9bdc06e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9bfc0abcd0 0x7f9bfc1ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bfc0c5fb0 0x7f9bfc1de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 --2- 172.21.15.129:0/514218707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bfc06e490 0x7f9bfc0bffa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 >> 172.21.15.129:0/514218707 conn(0x7f9bfc0a86b0 msgr2=0x7f9bfc0a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:43.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 shutdown_connections 2024-04-25T01:05:43.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.134+0000 7f9c00e9a640 1 -- 172.21.15.129:0/514218707 wait complete. 2024-04-25T01:05:43.142 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:05:43.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.248+0000 7fdccc2ba640 1 Processor -- start 2024-04-25T01:05:43.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.248+0000 7fdccc2ba640 1 -- start start 2024-04-25T01:05:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdccc2ba640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c6190 0x7fdcc40c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdccc2ba640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcc40c2aa0 0x7fdcc41566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdccc2ba640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc41530e0 con 0x7fdcc406d270 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdccc2ba640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc40c2f60 con 0x7fdcc40c2aa0 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdccc2ba640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc4153360 con 0x7fdcc40c6190 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcc40c2aa0 0x7fdcc41566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fdcc40c2aa0 0x7fdcc41566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55698/0 (socket says 172.21.15.129:55698) 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcc9855640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fdcc406d270 0x7fdcc40abcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:55470/0 (socket says 172.21.15.129:55470) 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 -- 172.21.15.129:0/353709483 learned_addr learned my addr 172.21.15.129:0/353709483 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca056640 1 --2- 172.21.15.129:0/353709483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c6190 0x7fdcc40c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 -- 172.21.15.129:0/353709483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c6190 msgr2=0x7fdcc40c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 --2- 172.21.15.129:0/353709483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c6190 0x7fdcc40c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 -- 172.21.15.129:0/353709483 >> v1:172.21.15.129:6789/0 conn(0x7fdcc406d270 legacy=0x7fdcc40abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcc9054640 1 -- 172.21.15.129:0/353709483 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737400614 0 0) 0x7fdcb4001110 con 0x7fdcc406d270 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.249+0000 7fdcca857640 1 -- 172.21.15.129:0/353709483 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc4153640 con 0x7fdcc40c2aa0 2024-04-25T01:05:43.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.250+0000 7fdcca857640 1 --2- 172.21.15.129:0/353709483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c2aa0 0x7fdcc41566d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009e50 tx=0x7fdcbc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8afc98bd71be6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:43.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.250+0000 7fdcc9054640 1 -- 172.21.15.129:0/353709483 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc036070 con 0x7fdcc40c2aa0 2024-04-25T01:05:43.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.250+0000 7fdcc9054640 1 -- 172.21.15.129:0/353709483 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc031600 con 0x7fdcc40c2aa0 2024-04-25T01:05:43.247 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.250+0000 7fdcc9054640 1 -- 172.21.15.129:0/353709483 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0319d0 con 0x7fdcc40c2aa0 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 -- 172.21.15.129:0/353709483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c2aa0 msgr2=0x7fdcc41566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/353709483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c2aa0 0x7fdcc41566d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009e50 tx=0x7fdcbc02a710 comp rx=0 tx=0).stop 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 -- 172.21.15.129:0/353709483 shutdown_connections 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/353709483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc40c2aa0 0x7fdcc41566d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/353709483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c6190 0x7fdcc40c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 -- 172.21.15.129:0/353709483 >> 172.21.15.129:0/353709483 conn(0x7fdcc40a86b0 msgr2=0x7fdcc40a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 -- 172.21.15.129:0/353709483 shutdown_connections 2024-04-25T01:05:43.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.251+0000 7fdccc2ba640 1 -- 172.21.15.129:0/353709483 wait complete. 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.252+0000 7fdccc2ba640 1 Processor -- start 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.252+0000 7fdccc2ba640 1 -- start start 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 0x7fdcc414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c2aa0 0x7fdcc4147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c6190 0x7fdcc41480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40ac550 con 0x7fdcc406d270 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40ac3d0 con 0x7fdcc40c6190 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc4153e40 con 0x7fdcc40c2aa0 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 0x7fdcc414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 0x7fdcc414bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55704/0 (socket says 172.21.15.129:55704) 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 -- 172.21.15.129:0/761160514 learned_addr learned my addr 172.21.15.129:0/761160514 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcc9855640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c2aa0 0x7fdcc4147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca857640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c6190 0x7fdcc41480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 -- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c2aa0 msgr2=0x7fdcc4147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c2aa0 0x7fdcc4147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 -- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c6190 msgr2=0x7fdcc41480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c6190 0x7fdcc41480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 -- 172.21.15.129:0/761160514 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc4148620 con 0x7fdcc406d270 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca857640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c6190 0x7fdcc41480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcca056640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 0x7fdcc414bcc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000e8a0 tx=0x7fdcc000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0017070 con 0x7fdcc406d270 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc000c940 con 0x7fdcc406d270 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000cd30 con 0x7fdcc406d270 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41488b0 con 0x7fdcc406d270 2024-04-25T01:05:43.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.253+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc414c200 con 0x7fdcc406d270 2024-04-25T01:05:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.255+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc4075550 con 0x7fdcc406d270 2024-04-25T01:05:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.255+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcc001a070 con 0x7fdcc406d270 2024-04-25T01:05:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.255+0000 7fdcbaffd640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca806be30 0x7fdca806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:05:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.256+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdcc008af90 con 0x7fdcc406d270 2024-04-25T01:05:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.256+0000 7fdcc9855640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca806be30 0x7fdca806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:05:43.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.256+0000 7fdcc9855640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca806be30 0x7fdca806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4150950 tx=0x7fdcb4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:05:43.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.260+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdcc0058f80 con 0x7fdcc406d270 2024-04-25T01:05:43.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.468+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdcc41e8a10 con 0x7fdcc406d270 2024-04-25T01:05:43.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.469+0000 7fdcbaffd640 1 -- 172.21.15.129:0/761160514 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fdcc005d280 con 0x7fdcc406d270 2024-04-25T01:05:43.464 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca806be30 msgr2=0x7fdca806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca806be30 0x7fdca806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4150950 tx=0x7fdcb4007b50 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 msgr2=0x7fdcc414bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 0x7fdcc414bcc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000e8a0 tx=0x7fdcc000ed60 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 shutdown_connections 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fdca806be30 0x7fdca806e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdcc40c2aa0 0x7fdcc4147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdcc40c6190 0x7fdcc41480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 --2- 172.21.15.129:0/761160514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc406d270 0x7fdcc414bcc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 >> 172.21.15.129:0/761160514 conn(0x7fdcc40a86b0 msgr2=0x7fdcc40a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 shutdown_connections 2024-04-25T01:05:43.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:05:43.471+0000 7fdccc2ba640 1 -- 172.21.15.129:0/761160514 wait complete. 2024-04-25T01:05:43.477 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi129.front.sepia.ceph.com... 2024-04-25T01:05:43.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:05:43.477 DEBUG:teuthology.orchestra.run.smithi129:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T01:05:43.501 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:05:43.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[44877]: fuse finished with error 0 and tester_r 0 2024-04-25T01:05:43.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 376298 15.93 MB/sec execute 24 sec latency 28.235 ms 2024-04-25T01:05:44.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 379775 16.06 MB/sec execute 25 sec latency 21.681 ms 2024-04-25T01:05:45.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 383082 16.12 MB/sec execute 26 sec latency 17.953 ms 2024-04-25T01:05:46.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 386150 16.06 MB/sec execute 27 sec latency 20.664 ms 2024-04-25T01:05:47.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 389154 15.99 MB/sec execute 28 sec latency 13.124 ms 2024-04-25T01:05:48.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 392397 16.03 MB/sec execute 29 sec latency 25.229 ms 2024-04-25T01:05:49.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:05:49.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:05:49.603 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T01:05:49.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:05:49.619 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-04-25T01:05:49.674 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:05:49.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:05:49.675 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:05:49.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:05:49.741 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-04-25T01:05:49.842 DEBUG:teuthology.parallel:result is None 2024-04-25T01:05:49.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 395276 16.11 MB/sec execute 30 sec latency 17.802 ms 2024-04-25T01:05:50.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 397513 16.04 MB/sec execute 31 sec latency 279.146 ms 2024-04-25T01:05:51.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 399645 15.76 MB/sec execute 32 sec latency 31.360 ms 2024-04-25T01:05:52.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 402551 15.81 MB/sec execute 33 sec latency 29.672 ms 2024-04-25T01:05:53.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 405852 15.90 MB/sec execute 34 sec latency 35.178 ms 2024-04-25T01:05:54.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 408892 15.87 MB/sec execute 35 sec latency 27.188 ms 2024-04-25T01:05:55.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 412292 15.96 MB/sec execute 36 sec latency 23.796 ms 2024-04-25T01:05:56.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 415707 16.03 MB/sec execute 37 sec latency 21.677 ms 2024-04-25T01:05:57.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 419179 16.11 MB/sec execute 38 sec latency 18.947 ms 2024-04-25T01:05:58.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 422589 16.19 MB/sec execute 39 sec latency 23.182 ms 2024-04-25T01:05:59.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 426049 16.26 MB/sec execute 40 sec latency 18.692 ms 2024-04-25T01:06:00.206 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:06:00.209 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:06:00.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 429337 16.30 MB/sec execute 41 sec latency 42.027 ms 2024-04-25T01:06:01.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 432591 16.29 MB/sec execute 42 sec latency 15.850 ms 2024-04-25T01:06:02.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 435974 16.32 MB/sec execute 43 sec latency 28.655 ms 2024-04-25T01:06:03.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 439190 16.34 MB/sec execute 44 sec latency 21.429 ms 2024-04-25T01:06:04.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 442190 16.30 MB/sec execute 45 sec latency 13.343 ms 2024-04-25T01:06:05.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 445026 16.33 MB/sec execute 46 sec latency 23.707 ms 2024-04-25T01:06:06.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 448204 16.32 MB/sec execute 47 sec latency 21.400 ms 2024-04-25T01:06:07.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 451759 16.38 MB/sec execute 48 sec latency 28.803 ms 2024-04-25T01:06:08.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 454964 16.41 MB/sec execute 49 sec latency 22.924 ms 2024-04-25T01:06:09.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 458344 16.45 MB/sec execute 50 sec latency 35.102 ms 2024-04-25T01:06:10.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 462129 16.91 MB/sec execute 51 sec latency 39.399 ms 2024-04-25T01:06:11.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 465680 16.95 MB/sec execute 52 sec latency 17.617 ms 2024-04-25T01:06:12.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 469277 16.99 MB/sec execute 53 sec latency 11.970 ms 2024-04-25T01:06:13.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 472743 17.03 MB/sec execute 54 sec latency 37.903 ms 2024-04-25T01:06:14.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 476162 17.05 MB/sec execute 55 sec latency 11.692 ms 2024-04-25T01:06:15.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 478625 16.91 MB/sec execute 56 sec latency 72.367 ms 2024-04-25T01:06:16.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 479055 16.70 MB/sec execute 57 sec latency 315.099 ms 2024-04-25T01:06:17.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 480239 16.52 MB/sec execute 58 sec latency 49.501 ms 2024-04-25T01:06:18.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 483606 16.57 MB/sec execute 59 sec latency 19.258 ms 2024-04-25T01:06:19.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 487015 16.61 MB/sec execute 60 sec latency 20.440 ms 2024-04-25T01:06:20.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 490457 16.63 MB/sec execute 61 sec latency 35.847 ms 2024-04-25T01:06:21.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 493858 16.67 MB/sec execute 62 sec latency 20.517 ms 2024-04-25T01:06:22.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 497060 16.64 MB/sec execute 63 sec latency 27.660 ms 2024-04-25T01:06:23.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 500672 16.69 MB/sec execute 64 sec latency 22.873 ms 2024-04-25T01:06:24.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 503873 16.70 MB/sec execute 65 sec latency 15.112 ms 2024-04-25T01:06:25.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 506938 16.66 MB/sec execute 66 sec latency 37.954 ms 2024-04-25T01:06:26.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 510370 16.69 MB/sec execute 67 sec latency 11.628 ms 2024-04-25T01:06:27.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 513284 16.70 MB/sec execute 68 sec latency 18.310 ms 2024-04-25T01:06:28.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 516279 16.69 MB/sec execute 69 sec latency 28.100 ms 2024-04-25T01:06:29.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 519652 16.71 MB/sec execute 70 sec latency 31.716 ms 2024-04-25T01:06:30.278 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:06:30.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:06:30.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 522958 16.74 MB/sec execute 71 sec latency 29.754 ms 2024-04-25T01:06:31.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 526450 16.77 MB/sec execute 72 sec latency 25.746 ms 2024-04-25T01:06:32.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 529974 16.80 MB/sec execute 73 sec latency 22.652 ms 2024-04-25T01:06:33.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 533421 16.83 MB/sec execute 74 sec latency 12.391 ms 2024-04-25T01:06:34.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 536822 16.86 MB/sec execute 75 sec latency 39.598 ms 2024-04-25T01:06:35.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 540270 16.88 MB/sec execute 76 sec latency 13.996 ms 2024-04-25T01:06:36.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 543655 16.90 MB/sec execute 77 sec latency 17.025 ms 2024-04-25T01:06:37.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 544340 16.71 MB/sec execute 78 sec latency 301.854 ms 2024-04-25T01:06:38.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 545928 16.58 MB/sec execute 79 sec latency 154.577 ms 2024-04-25T01:06:39.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 549460 16.61 MB/sec execute 80 sec latency 12.065 ms 2024-04-25T01:06:40.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 552143 16.62 MB/sec execute 81 sec latency 76.515 ms 2024-04-25T01:06:41.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 555967 16.65 MB/sec execute 82 sec latency 14.615 ms 2024-04-25T01:06:42.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 559235 16.68 MB/sec execute 83 sec latency 24.174 ms 2024-04-25T01:06:43.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 562436 16.67 MB/sec execute 84 sec latency 31.076 ms 2024-04-25T01:06:44.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 565773 16.69 MB/sec execute 85 sec latency 22.285 ms 2024-04-25T01:06:45.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 569134 16.71 MB/sec execute 86 sec latency 35.766 ms 2024-04-25T01:06:46.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 572627 16.74 MB/sec execute 87 sec latency 12.158 ms 2024-04-25T01:06:47.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 576100 16.76 MB/sec execute 88 sec latency 32.956 ms 2024-04-25T01:06:48.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 579552 16.79 MB/sec execute 89 sec latency 28.804 ms 2024-04-25T01:06:49.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 582913 16.81 MB/sec execute 90 sec latency 15.292 ms 2024-04-25T01:06:50.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 586435 16.84 MB/sec execute 91 sec latency 35.906 ms 2024-04-25T01:06:51.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 589782 16.85 MB/sec execute 92 sec latency 37.904 ms 2024-04-25T01:06:52.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 593161 16.87 MB/sec execute 93 sec latency 19.329 ms 2024-04-25T01:06:53.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 596176 16.84 MB/sec execute 94 sec latency 20.648 ms 2024-04-25T01:06:54.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 599641 16.86 MB/sec execute 95 sec latency 19.473 ms 2024-04-25T01:06:55.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 602723 16.84 MB/sec execute 96 sec latency 17.983 ms 2024-04-25T01:06:56.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 605612 16.84 MB/sec execute 97 sec latency 26.689 ms 2024-04-25T01:06:57.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 608480 16.83 MB/sec execute 98 sec latency 22.543 ms 2024-04-25T01:06:58.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 610261 16.76 MB/sec execute 99 sec latency 309.034 ms 2024-04-25T01:06:59.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 611334 16.66 MB/sec execute 100 sec latency 139.999 ms 2024-04-25T01:07:00.356 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:00.358 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:00.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 614714 16.68 MB/sec execute 101 sec latency 21.579 ms 2024-04-25T01:07:01.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 618193 16.70 MB/sec execute 102 sec latency 20.543 ms 2024-04-25T01:07:02.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 621537 16.72 MB/sec execute 103 sec latency 21.011 ms 2024-04-25T01:07:03.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 624530 16.69 MB/sec execute 104 sec latency 12.893 ms 2024-04-25T01:07:04.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 627995 16.71 MB/sec execute 105 sec latency 28.933 ms 2024-04-25T01:07:05.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 631020 16.68 MB/sec execute 106 sec latency 21.870 ms 2024-04-25T01:07:06.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 633930 16.69 MB/sec execute 107 sec latency 31.760 ms 2024-04-25T01:07:07.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 637121 16.71 MB/sec execute 108 sec latency 15.165 ms 2024-04-25T01:07:08.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 640246 16.70 MB/sec execute 109 sec latency 23.746 ms 2024-04-25T01:07:09.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 643589 16.71 MB/sec execute 110 sec latency 39.020 ms 2024-04-25T01:07:10.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 646780 16.73 MB/sec execute 111 sec latency 87.757 ms 2024-04-25T01:07:11.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 650232 16.75 MB/sec execute 112 sec latency 28.451 ms 2024-04-25T01:07:12.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 653532 16.76 MB/sec execute 113 sec latency 21.505 ms 2024-04-25T01:07:13.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 655355 16.67 MB/sec execute 114 sec latency 221.876 ms 2024-04-25T01:07:14.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 657115 16.64 MB/sec execute 115 sec latency 92.363 ms 2024-04-25T01:07:15.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 659474 16.57 MB/sec execute 116 sec latency 110.217 ms 2024-04-25T01:07:16.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 662593 16.58 MB/sec execute 117 sec latency 19.680 ms 2024-04-25T01:07:17.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 665603 16.59 MB/sec execute 118 sec latency 26.416 ms 2024-04-25T01:07:18.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 668758 16.59 MB/sec execute 119 sec latency 24.288 ms 2024-04-25T01:07:19.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 671954 16.60 MB/sec execute 120 sec latency 25.999 ms 2024-04-25T01:07:20.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 675360 16.61 MB/sec execute 121 sec latency 57.827 ms 2024-04-25T01:07:21.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 678744 16.63 MB/sec execute 122 sec latency 25.632 ms 2024-04-25T01:07:22.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 682074 16.65 MB/sec execute 123 sec latency 27.344 ms 2024-04-25T01:07:23.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 685392 16.66 MB/sec execute 124 sec latency 27.959 ms 2024-04-25T01:07:24.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 688370 16.64 MB/sec execute 125 sec latency 21.909 ms 2024-04-25T01:07:25.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 691350 16.62 MB/sec execute 126 sec latency 15.830 ms 2024-04-25T01:07:26.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 694100 16.62 MB/sec execute 127 sec latency 15.164 ms 2024-04-25T01:07:27.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 697108 16.62 MB/sec execute 128 sec latency 18.850 ms 2024-04-25T01:07:28.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 700324 16.62 MB/sec execute 129 sec latency 20.322 ms 2024-04-25T01:07:29.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 703812 16.64 MB/sec execute 130 sec latency 33.161 ms 2024-04-25T01:07:30.495 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:30.497 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:30.681 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:07:30.689+0000 7f224640f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41535) UID: 0 2024-04-25T01:07:30.682 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:07:30.689+0000 7ff199406640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41535) UID: 0 2024-04-25T01:07:30.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 707215 16.66 MB/sec execute 131 sec latency 23.204 ms 2024-04-25T01:07:31.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 710510 16.67 MB/sec execute 132 sec latency 30.872 ms 2024-04-25T01:07:32.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 713749 16.68 MB/sec execute 133 sec latency 28.079 ms 2024-04-25T01:07:33.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 716347 16.63 MB/sec execute 134 sec latency 22.019 ms 2024-04-25T01:07:34.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 717421 16.58 MB/sec execute 135 sec latency 383.417 ms 2024-04-25T01:07:35.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 718483 16.48 MB/sec execute 136 sec latency 51.540 ms 2024-04-25T01:07:36.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 721291 16.49 MB/sec execute 137 sec latency 45.547 ms 2024-04-25T01:07:37.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 724533 16.50 MB/sec execute 138 sec latency 35.533 ms 2024-04-25T01:07:38.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 727402 16.48 MB/sec execute 139 sec latency 40.928 ms 2024-04-25T01:07:39.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 730382 16.46 MB/sec execute 140 sec latency 17.041 ms 2024-04-25T01:07:40.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 733025 16.46 MB/sec execute 141 sec latency 31.996 ms 2024-04-25T01:07:41.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 735828 16.45 MB/sec execute 142 sec latency 16.821 ms 2024-04-25T01:07:42.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 739182 16.46 MB/sec execute 143 sec latency 25.009 ms 2024-04-25T01:07:43.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 742498 16.48 MB/sec execute 144 sec latency 28.203 ms 2024-04-25T01:07:44.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 745746 16.49 MB/sec execute 145 sec latency 13.957 ms 2024-04-25T01:07:45.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 748155 16.44 MB/sec execute 146 sec latency 59.192 ms 2024-04-25T01:07:46.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 750742 16.44 MB/sec execute 147 sec latency 20.216 ms 2024-04-25T01:07:47.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 753932 16.44 MB/sec execute 148 sec latency 36.643 ms 2024-04-25T01:07:48.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 757241 16.45 MB/sec execute 149 sec latency 16.758 ms 2024-04-25T01:07:49.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 760674 16.46 MB/sec execute 150 sec latency 17.355 ms 2024-04-25T01:07:50.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 764147 16.48 MB/sec execute 151 sec latency 27.041 ms 2024-04-25T01:07:51.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 767623 16.50 MB/sec execute 152 sec latency 27.884 ms 2024-04-25T01:07:52.866 INFO:tasks.workunit.client.1.smithi171.stdout: 1 770965 16.51 MB/sec execute 153 sec latency 26.407 ms 2024-04-25T01:07:53.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 774340 16.53 MB/sec execute 154 sec latency 50.020 ms 2024-04-25T01:07:54.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 777582 16.54 MB/sec execute 155 sec latency 37.581 ms 2024-04-25T01:07:55.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 780350 16.51 MB/sec execute 156 sec latency 45.655 ms 2024-04-25T01:07:56.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 783118 16.49 MB/sec execute 157 sec latency 13.825 ms 2024-04-25T01:07:57.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 785789 16.49 MB/sec execute 158 sec latency 26.518 ms 2024-04-25T01:07:58.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 788906 16.49 MB/sec execute 159 sec latency 20.877 ms 2024-04-25T01:07:59.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 792094 16.51 MB/sec execute 160 sec latency 19.671 ms 2024-04-25T01:08:00.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 795108 16.49 MB/sec execute 161 sec latency 30.145 ms 2024-04-25T01:08:01.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 797668 16.46 MB/sec execute 162 sec latency 14.457 ms 2024-04-25T01:08:02.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 799900 16.45 MB/sec execute 163 sec latency 27.455 ms 2024-04-25T01:08:03.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 803031 16.45 MB/sec execute 164 sec latency 19.368 ms 2024-04-25T01:08:04.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 806201 16.47 MB/sec execute 165 sec latency 29.513 ms 2024-04-25T01:08:05.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 808720 16.44 MB/sec execute 166 sec latency 125.642 ms 2024-04-25T01:08:06.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 808722 16.34 MB/sec execute 167 sec latency 539.485 ms 2024-04-25T01:08:07.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 810041 16.29 MB/sec execute 168 sec latency 642.152 ms 2024-04-25T01:08:08.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 811706 16.23 MB/sec execute 169 sec latency 50.632 ms 2024-04-25T01:08:09.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 814113 16.22 MB/sec execute 170 sec latency 20.991 ms 2024-04-25T01:08:10.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 817193 16.23 MB/sec execute 171 sec latency 30.807 ms 2024-04-25T01:08:11.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 820450 16.24 MB/sec execute 172 sec latency 26.000 ms 2024-04-25T01:08:12.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 823115 16.22 MB/sec execute 173 sec latency 32.647 ms 2024-04-25T01:08:13.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 826138 16.21 MB/sec execute 174 sec latency 19.084 ms 2024-04-25T01:08:14.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 828876 16.21 MB/sec execute 175 sec latency 13.062 ms 2024-04-25T01:08:15.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 831588 16.20 MB/sec execute 176 sec latency 39.093 ms 2024-04-25T01:08:16.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 835000 16.22 MB/sec execute 177 sec latency 13.298 ms 2024-04-25T01:08:17.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 838127 16.23 MB/sec execute 178 sec latency 23.120 ms 2024-04-25T01:08:18.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 841204 16.22 MB/sec execute 179 sec latency 25.090 ms 2024-04-25T01:08:19.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 844063 16.21 MB/sec execute 180 sec latency 51.727 ms 2024-04-25T01:08:20.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 847111 16.21 MB/sec execute 181 sec latency 21.138 ms 2024-04-25T01:08:21.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 850082 16.22 MB/sec execute 182 sec latency 35.550 ms 2024-04-25T01:08:22.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 852933 16.21 MB/sec execute 183 sec latency 28.549 ms 2024-04-25T01:08:23.869 INFO:tasks.workunit.client.1.smithi171.stdout: 1 856251 16.23 MB/sec execute 184 sec latency 26.026 ms 2024-04-25T01:08:24.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 859493 16.24 MB/sec execute 185 sec latency 22.374 ms 2024-04-25T01:08:25.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 862397 16.22 MB/sec execute 186 sec latency 40.791 ms 2024-04-25T01:08:26.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 863411 16.17 MB/sec execute 187 sec latency 256.080 ms 2024-04-25T01:08:27.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 865235 16.12 MB/sec execute 188 sec latency 332.972 ms 2024-04-25T01:08:28.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 867093 16.10 MB/sec execute 189 sec latency 49.378 ms 2024-04-25T01:08:29.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 870217 16.12 MB/sec execute 190 sec latency 17.292 ms 2024-04-25T01:08:30.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 873346 16.12 MB/sec execute 191 sec latency 51.360 ms 2024-04-25T01:08:31.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 876144 16.10 MB/sec execute 192 sec latency 28.785 ms 2024-04-25T01:08:32.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 878703 16.09 MB/sec execute 193 sec latency 38.730 ms 2024-04-25T01:08:33.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 881645 16.09 MB/sec execute 194 sec latency 16.003 ms 2024-04-25T01:08:34.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 884706 16.10 MB/sec execute 195 sec latency 25.308 ms 2024-04-25T01:08:35.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 887991 16.11 MB/sec execute 196 sec latency 32.145 ms 2024-04-25T01:08:36.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 891182 16.12 MB/sec execute 197 sec latency 32.698 ms 2024-04-25T01:08:37.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 893607 16.08 MB/sec execute 198 sec latency 38.376 ms 2024-04-25T01:08:38.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 896500 16.09 MB/sec execute 199 sec latency 23.435 ms 2024-04-25T01:08:39.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 899202 16.08 MB/sec execute 200 sec latency 17.668 ms 2024-04-25T01:08:40.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 902425 16.09 MB/sec execute 201 sec latency 20.889 ms 2024-04-25T01:08:41.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 905364 16.10 MB/sec execute 202 sec latency 96.952 ms 2024-04-25T01:08:42.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 906035 16.03 MB/sec execute 203 sec latency 219.575 ms 2024-04-25T01:08:43.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 909047 16.04 MB/sec execute 204 sec latency 31.897 ms 2024-04-25T01:08:44.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 912213 16.03 MB/sec execute 205 sec latency 34.983 ms 2024-04-25T01:08:45.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 914978 16.02 MB/sec execute 206 sec latency 31.364 ms 2024-04-25T01:08:46.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 917658 16.10 MB/sec execute 207 sec latency 33.887 ms 2024-04-25T01:08:47.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 921285 16.12 MB/sec execute 208 sec latency 11.873 ms 2024-04-25T01:08:48.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 924473 16.12 MB/sec execute 209 sec latency 39.586 ms 2024-04-25T01:08:49.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 928003 16.14 MB/sec execute 210 sec latency 8.059 ms 2024-04-25T01:08:50.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 931211 16.15 MB/sec execute 211 sec latency 34.545 ms 2024-04-25T01:08:51.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 934715 16.16 MB/sec execute 212 sec latency 17.646 ms 2024-04-25T01:08:52.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 938254 16.18 MB/sec execute 213 sec latency 11.753 ms 2024-04-25T01:08:53.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 940928 16.16 MB/sec execute 214 sec latency 33.243 ms 2024-04-25T01:08:54.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 944166 16.16 MB/sec execute 215 sec latency 13.524 ms 2024-04-25T01:08:55.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 947560 16.17 MB/sec execute 216 sec latency 21.108 ms 2024-04-25T01:08:56.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 950078 16.17 MB/sec execute 217 sec latency 32.897 ms 2024-04-25T01:08:57.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 953417 16.17 MB/sec execute 218 sec latency 12.378 ms 2024-04-25T01:08:58.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 956736 16.18 MB/sec execute 219 sec latency 18.925 ms 2024-04-25T01:08:59.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 960007 16.19 MB/sec execute 220 sec latency 28.424 ms 2024-04-25T01:09:00.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 963235 16.20 MB/sec execute 221 sec latency 74.848 ms 2024-04-25T01:09:01.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 966579 16.21 MB/sec execute 222 sec latency 21.113 ms 2024-04-25T01:09:02.869 INFO:tasks.workunit.client.1.smithi171.stdout: 1 967053 16.14 MB/sec execute 223 sec latency 488.217 ms 2024-04-25T01:09:03.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 968725 16.10 MB/sec execute 224 sec latency 519.697 ms 2024-04-25T01:09:04.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 970186 16.08 MB/sec execute 225 sec latency 50.356 ms 2024-04-25T01:09:05.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 972521 16.05 MB/sec execute 226 sec latency 27.539 ms 2024-04-25T01:09:06.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 974623 16.03 MB/sec execute 227 sec latency 47.636 ms 2024-04-25T01:09:07.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 977580 16.04 MB/sec execute 228 sec latency 21.041 ms 2024-04-25T01:09:08.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 980770 16.05 MB/sec execute 229 sec latency 23.057 ms 2024-04-25T01:09:09.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 983556 16.04 MB/sec execute 230 sec latency 53.905 ms 2024-04-25T01:09:10.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 985800 16.02 MB/sec execute 231 sec latency 32.604 ms 2024-04-25T01:09:11.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 988553 16.01 MB/sec execute 232 sec latency 28.450 ms 2024-04-25T01:09:12.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 991805 16.02 MB/sec execute 233 sec latency 35.872 ms 2024-04-25T01:09:13.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 994991 16.03 MB/sec execute 234 sec latency 14.068 ms 2024-04-25T01:09:14.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 998100 16.03 MB/sec execute 235 sec latency 25.847 ms 2024-04-25T01:09:15.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1000814 16.01 MB/sec execute 236 sec latency 31.520 ms 2024-04-25T01:09:16.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1003357 16.02 MB/sec execute 237 sec latency 33.821 ms 2024-04-25T01:09:17.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1006482 16.02 MB/sec execute 238 sec latency 21.087 ms 2024-04-25T01:09:18.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1009690 16.02 MB/sec execute 239 sec latency 13.787 ms 2024-04-25T01:09:19.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1012960 16.03 MB/sec execute 240 sec latency 23.136 ms 2024-04-25T01:09:20.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1016306 16.04 MB/sec execute 241 sec latency 17.718 ms 2024-04-25T01:09:21.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1019242 16.04 MB/sec execute 242 sec latency 32.196 ms 2024-04-25T01:09:22.869 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1021996 16.02 MB/sec execute 243 sec latency 29.148 ms 2024-04-25T01:09:23.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1024544 16.03 MB/sec execute 244 sec latency 17.337 ms 2024-04-25T01:09:24.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1027433 16.02 MB/sec execute 245 sec latency 15.294 ms 2024-04-25T01:09:25.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1030623 16.03 MB/sec execute 246 sec latency 43.891 ms 2024-04-25T01:09:26.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1033743 16.04 MB/sec execute 247 sec latency 24.700 ms 2024-04-25T01:09:27.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1036233 16.01 MB/sec execute 248 sec latency 33.708 ms 2024-04-25T01:09:28.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1038915 16.01 MB/sec execute 249 sec latency 35.636 ms 2024-04-25T01:09:29.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1041844 16.01 MB/sec execute 250 sec latency 23.554 ms 2024-04-25T01:09:30.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1045008 16.02 MB/sec execute 251 sec latency 35.509 ms 2024-04-25T01:09:31.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1048258 16.03 MB/sec execute 252 sec latency 26.485 ms 2024-04-25T01:09:32.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1051498 16.03 MB/sec execute 253 sec latency 24.794 ms 2024-04-25T01:09:33.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1054356 16.02 MB/sec execute 254 sec latency 22.752 ms 2024-04-25T01:09:34.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1056747 16.01 MB/sec execute 255 sec latency 40.308 ms 2024-04-25T01:09:35.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1059592 16.01 MB/sec execute 256 sec latency 53.467 ms 2024-04-25T01:09:36.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1062824 16.02 MB/sec execute 257 sec latency 21.301 ms 2024-04-25T01:09:37.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1066001 16.03 MB/sec execute 258 sec latency 23.626 ms 2024-04-25T01:09:38.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1068167 15.99 MB/sec execute 259 sec latency 65.114 ms 2024-04-25T01:09:39.867 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1070648 16.00 MB/sec execute 260 sec latency 71.209 ms 2024-04-25T01:09:40.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1073378 15.99 MB/sec execute 261 sec latency 50.601 ms 2024-04-25T01:09:41.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1076554 16.00 MB/sec execute 262 sec latency 21.948 ms 2024-04-25T01:09:42.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1079261 15.99 MB/sec execute 263 sec latency 112.667 ms 2024-04-25T01:09:43.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1079961 15.95 MB/sec execute 264 sec latency 536.466 ms 2024-04-25T01:09:44.869 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1080306 15.89 MB/sec execute 265 sec latency 383.489 ms 2024-04-25T01:09:45.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1080758 15.83 MB/sec execute 266 sec latency 80.422 ms 2024-04-25T01:09:46.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1083548 15.84 MB/sec execute 267 sec latency 41.086 ms 2024-04-25T01:09:47.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1086399 15.83 MB/sec execute 268 sec latency 13.485 ms 2024-04-25T01:09:48.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1089052 15.82 MB/sec execute 269 sec latency 38.659 ms 2024-04-25T01:09:49.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1091583 15.82 MB/sec execute 270 sec latency 43.800 ms 2024-04-25T01:09:50.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1094775 15.82 MB/sec execute 271 sec latency 54.780 ms 2024-04-25T01:09:51.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1098018 15.83 MB/sec execute 272 sec latency 60.475 ms 2024-04-25T01:09:52.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1101157 15.84 MB/sec execute 273 sec latency 17.166 ms 2024-04-25T01:09:53.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1103910 15.83 MB/sec execute 274 sec latency 32.166 ms 2024-04-25T01:09:54.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1106895 15.82 MB/sec execute 275 sec latency 21.582 ms 2024-04-25T01:09:55.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1109455 15.82 MB/sec execute 276 sec latency 20.988 ms 2024-04-25T01:09:56.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1112671 15.83 MB/sec execute 277 sec latency 27.380 ms 2024-04-25T01:09:57.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1116059 15.84 MB/sec execute 278 sec latency 25.337 ms 2024-04-25T01:09:58.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1119493 15.85 MB/sec execute 279 sec latency 16.262 ms 2024-04-25T01:09:59.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1122741 15.86 MB/sec execute 280 sec latency 30.666 ms 2024-04-25T01:10:00.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1125791 15.85 MB/sec execute 281 sec latency 30.940 ms 2024-04-25T01:10:01.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1128548 15.84 MB/sec execute 282 sec latency 67.461 ms 2024-04-25T01:10:02.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1131099 15.85 MB/sec execute 283 sec latency 27.937 ms 2024-04-25T01:10:03.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1133579 15.84 MB/sec execute 284 sec latency 209.074 ms 2024-04-25T01:10:04.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1135089 15.80 MB/sec execute 285 sec latency 208.113 ms 2024-04-25T01:10:05.868 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1136876 15.79 MB/sec execute 286 sec latency 47.860 ms 2024-04-25T01:10:06.870 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1139955 15.79 MB/sec execute 287 sec latency 21.258 ms 2024-04-25T01:10:07.870 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1142961 15.79 MB/sec execute 288 sec latency 18.465 ms 2024-04-25T01:10:08.870 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1145570 15.78 MB/sec execute 289 sec latency 30.411 ms 2024-04-25T01:10:09.870 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1148476 15.78 MB/sec execute 290 sec latency 29.262 ms 2024-04-25T01:10:10.874 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1151625 15.79 MB/sec execute 291 sec latency 28.782 ms 2024-04-25T01:10:11.874 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1154959 15.80 MB/sec execute 292 sec latency 25.974 ms 2024-04-25T01:10:12.874 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1158170 15.81 MB/sec execute 293 sec latency 13.481 ms 2024-04-25T01:10:13.875 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1161263 15.80 MB/sec execute 294 sec latency 34.580 ms 2024-04-25T01:10:14.875 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1164352 15.80 MB/sec execute 295 sec latency 19.588 ms 2024-04-25T01:10:15.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1167363 15.80 MB/sec execute 296 sec latency 22.698 ms 2024-04-25T01:10:16.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:16.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:16.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1169818 15.80 MB/sec execute 297 sec latency 35.631 ms 2024-04-25T01:10:17.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1172899 15.80 MB/sec execute 298 sec latency 32.116 ms 2024-04-25T01:10:18.875 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1176341 15.81 MB/sec execute 299 sec latency 28.721 ms 2024-04-25T01:10:19.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1179649 15.82 MB/sec execute 300 sec latency 20.275 ms 2024-04-25T01:10:20.875 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1183024 15.83 MB/sec execute 301 sec latency 30.064 ms 2024-04-25T01:10:21.875 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1186282 15.84 MB/sec execute 302 sec latency 34.546 ms 2024-04-25T01:10:22.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1189492 15.84 MB/sec execute 303 sec latency 19.738 ms 2024-04-25T01:10:23.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1192632 15.84 MB/sec execute 304 sec latency 22.065 ms 2024-04-25T01:10:24.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1195750 15.84 MB/sec execute 305 sec latency 18.490 ms 2024-04-25T01:10:25.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1198436 15.84 MB/sec execute 306 sec latency 29.380 ms 2024-04-25T01:10:26.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1201387 15.84 MB/sec execute 307 sec latency 35.748 ms 2024-04-25T01:10:27.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1204743 15.85 MB/sec execute 308 sec latency 36.305 ms 2024-04-25T01:10:28.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1208089 15.86 MB/sec execute 309 sec latency 25.147 ms 2024-04-25T01:10:29.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1211490 15.87 MB/sec execute 310 sec latency 23.092 ms 2024-04-25T01:10:30.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1211999 15.82 MB/sec execute 311 sec latency 586.584 ms 2024-04-25T01:10:31.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1213997 15.80 MB/sec execute 312 sec latency 310.304 ms 2024-04-25T01:10:32.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1214941 15.77 MB/sec execute 313 sec latency 57.412 ms 2024-04-25T01:10:33.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1217935 15.77 MB/sec execute 314 sec latency 21.676 ms 2024-04-25T01:10:34.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1220587 15.76 MB/sec execute 315 sec latency 25.442 ms 2024-04-25T01:10:35.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1222960 15.75 MB/sec execute 316 sec latency 41.966 ms 2024-04-25T01:10:36.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1225909 15.76 MB/sec execute 317 sec latency 52.311 ms 2024-04-25T01:10:37.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1228984 15.76 MB/sec execute 318 sec latency 26.529 ms 2024-04-25T01:10:38.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1231830 15.76 MB/sec execute 319 sec latency 18.919 ms 2024-04-25T01:10:39.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1234653 15.75 MB/sec execute 320 sec latency 12.966 ms 2024-04-25T01:10:40.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1237166 15.75 MB/sec execute 321 sec latency 18.832 ms 2024-04-25T01:10:41.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1240085 15.75 MB/sec execute 322 sec latency 109.438 ms 2024-04-25T01:10:42.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1243365 15.76 MB/sec execute 323 sec latency 31.587 ms 2024-04-25T01:10:43.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1246649 15.77 MB/sec execute 324 sec latency 14.236 ms 2024-04-25T01:10:44.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1249356 15.76 MB/sec execute 325 sec latency 31.357 ms 2024-04-25T01:10:45.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1250929 15.73 MB/sec execute 326 sec latency 290.031 ms 2024-04-25T01:10:46.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:46.495 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:46.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1253485 15.73 MB/sec execute 327 sec latency 333.386 ms 2024-04-25T01:10:47.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1254945 15.70 MB/sec execute 328 sec latency 39.347 ms 2024-04-25T01:10:48.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1257991 15.71 MB/sec execute 329 sec latency 24.927 ms 2024-04-25T01:10:49.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1261194 15.72 MB/sec execute 330 sec latency 32.286 ms 2024-04-25T01:10:50.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1264405 15.72 MB/sec execute 331 sec latency 26.593 ms 2024-04-25T01:10:51.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1267277 15.72 MB/sec execute 332 sec latency 18.765 ms 2024-04-25T01:10:52.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1270264 15.71 MB/sec execute 333 sec latency 28.300 ms 2024-04-25T01:10:53.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1272841 15.71 MB/sec execute 334 sec latency 32.945 ms 2024-04-25T01:10:54.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1275740 15.71 MB/sec execute 335 sec latency 30.334 ms 2024-04-25T01:10:55.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1279088 15.72 MB/sec execute 336 sec latency 25.674 ms 2024-04-25T01:10:56.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1282353 15.73 MB/sec execute 337 sec latency 30.266 ms 2024-04-25T01:10:57.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1285806 15.74 MB/sec execute 338 sec latency 31.347 ms 2024-04-25T01:10:58.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1288947 15.74 MB/sec execute 339 sec latency 12.723 ms 2024-04-25T01:10:59.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1291993 15.74 MB/sec execute 340 sec latency 19.244 ms 2024-04-25T01:11:00.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1294311 15.73 MB/sec execute 341 sec latency 27.407 ms 2024-04-25T01:11:01.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1297430 15.73 MB/sec execute 342 sec latency 25.174 ms 2024-04-25T01:11:02.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1300705 15.74 MB/sec execute 343 sec latency 31.351 ms 2024-04-25T01:11:03.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1304024 15.74 MB/sec execute 344 sec latency 34.850 ms 2024-04-25T01:11:04.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1307123 15.75 MB/sec execute 345 sec latency 104.101 ms 2024-04-25T01:11:05.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1310080 15.75 MB/sec execute 346 sec latency 11.900 ms 2024-04-25T01:11:06.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1312858 15.74 MB/sec execute 347 sec latency 23.817 ms 2024-04-25T01:11:07.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1315407 15.74 MB/sec execute 348 sec latency 17.625 ms 2024-04-25T01:11:08.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1318359 15.74 MB/sec execute 349 sec latency 31.471 ms 2024-04-25T01:11:09.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1321765 15.75 MB/sec execute 350 sec latency 23.447 ms 2024-04-25T01:11:10.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1325182 15.76 MB/sec execute 351 sec latency 25.790 ms 2024-04-25T01:11:11.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1325629 15.71 MB/sec execute 352 sec latency 354.112 ms 2024-04-25T01:11:12.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1326332 15.68 MB/sec execute 353 sec latency 399.441 ms 2024-04-25T01:11:13.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1328644 15.68 MB/sec execute 354 sec latency 34.094 ms 2024-04-25T01:11:14.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1331985 15.69 MB/sec execute 355 sec latency 51.799 ms 2024-04-25T01:11:15.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1335078 15.68 MB/sec execute 356 sec latency 24.728 ms 2024-04-25T01:11:16.788 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:16.791 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:16.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1337738 15.67 MB/sec execute 357 sec latency 41.483 ms 2024-04-25T01:11:17.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1339956 15.67 MB/sec execute 358 sec latency 62.866 ms 2024-04-25T01:11:18.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1343040 15.67 MB/sec execute 359 sec latency 24.273 ms 2024-04-25T01:11:19.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1346234 15.68 MB/sec execute 360 sec latency 26.387 ms 2024-04-25T01:11:20.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1349212 15.68 MB/sec execute 361 sec latency 27.096 ms 2024-04-25T01:11:21.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1352212 15.68 MB/sec execute 362 sec latency 16.620 ms 2024-04-25T01:11:22.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1355336 15.68 MB/sec execute 363 sec latency 28.183 ms 2024-04-25T01:11:23.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1358152 15.68 MB/sec execute 364 sec latency 26.041 ms 2024-04-25T01:11:24.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1361019 15.68 MB/sec execute 365 sec latency 62.412 ms 2024-04-25T01:11:25.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1364367 15.69 MB/sec execute 366 sec latency 31.293 ms 2024-04-25T01:11:26.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1366614 15.68 MB/sec execute 367 sec latency 197.051 ms 2024-04-25T01:11:27.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1367602 15.65 MB/sec execute 368 sec latency 346.859 ms 2024-04-25T01:11:28.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1370733 15.66 MB/sec execute 369 sec latency 41.609 ms 2024-04-25T01:11:29.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1373698 15.65 MB/sec execute 370 sec latency 27.226 ms 2024-04-25T01:11:30.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1375934 15.69 MB/sec execute 371 sec latency 35.021 ms 2024-04-25T01:11:31.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1379381 15.70 MB/sec execute 372 sec latency 28.112 ms 2024-04-25T01:11:32.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1382923 15.71 MB/sec execute 373 sec latency 13.267 ms 2024-04-25T01:11:33.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1386425 15.71 MB/sec execute 374 sec latency 12.081 ms 2024-04-25T01:11:34.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1389763 15.72 MB/sec execute 375 sec latency 33.758 ms 2024-04-25T01:11:35.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1393052 15.73 MB/sec execute 376 sec latency 12.469 ms 2024-04-25T01:11:36.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1395743 15.72 MB/sec execute 377 sec latency 58.251 ms 2024-04-25T01:11:37.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1398734 15.72 MB/sec execute 378 sec latency 11.372 ms 2024-04-25T01:11:38.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1401309 15.72 MB/sec execute 379 sec latency 14.758 ms 2024-04-25T01:11:39.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1404115 15.72 MB/sec execute 380 sec latency 38.226 ms 2024-04-25T01:11:40.876 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1407445 15.73 MB/sec execute 381 sec latency 11.317 ms 2024-04-25T01:11:41.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1410505 15.73 MB/sec execute 382 sec latency 34.864 ms 2024-04-25T01:11:42.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1413458 15.73 MB/sec execute 383 sec latency 20.951 ms 2024-04-25T01:11:43.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1416068 15.72 MB/sec execute 384 sec latency 11.108 ms 2024-04-25T01:11:44.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1418613 15.71 MB/sec execute 385 sec latency 13.639 ms 2024-04-25T01:11:45.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1421753 15.72 MB/sec execute 386 sec latency 45.773 ms 2024-04-25T01:11:46.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1424987 15.73 MB/sec execute 387 sec latency 12.810 ms 2024-04-25T01:11:47.210 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:47.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:47.258 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:11:47.270+0000 7ff199406640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41621) UID: 0 2024-04-25T01:11:47.261 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:11:47.274+0000 7f224640f640 -1 Fail to open '/proc/41621/cmdline' error = (2) No such file or directory 2024-04-25T01:11:47.262 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:11:47.274+0000 7f224640f640 -1 received signal: Hangup from (PID: 41621) UID: 0 2024-04-25T01:11:47.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1425909 15.69 MB/sec execute 388 sec latency 238.006 ms 2024-04-25T01:11:48.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1426923 15.66 MB/sec execute 389 sec latency 365.791 ms 2024-04-25T01:11:49.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1428490 15.66 MB/sec execute 390 sec latency 58.874 ms 2024-04-25T01:11:50.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1430646 15.64 MB/sec execute 391 sec latency 25.783 ms 2024-04-25T01:11:51.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1432967 15.63 MB/sec execute 392 sec latency 33.967 ms 2024-04-25T01:11:52.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1436102 15.64 MB/sec execute 393 sec latency 26.858 ms 2024-04-25T01:11:53.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1439352 15.64 MB/sec execute 394 sec latency 41.798 ms 2024-04-25T01:11:54.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1442239 15.64 MB/sec execute 395 sec latency 31.131 ms 2024-04-25T01:11:55.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1444750 15.63 MB/sec execute 396 sec latency 15.175 ms 2024-04-25T01:11:56.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1446732 15.62 MB/sec execute 397 sec latency 72.172 ms 2024-04-25T01:11:57.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1449651 15.63 MB/sec execute 398 sec latency 73.342 ms 2024-04-25T01:11:58.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1451944 15.61 MB/sec execute 399 sec latency 28.126 ms 2024-04-25T01:11:59.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1453924 15.60 MB/sec execute 400 sec latency 34.644 ms 2024-04-25T01:12:00.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1457058 15.61 MB/sec execute 401 sec latency 37.783 ms 2024-04-25T01:12:01.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1460004 15.61 MB/sec execute 402 sec latency 29.158 ms 2024-04-25T01:12:02.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1462784 15.60 MB/sec execute 403 sec latency 29.505 ms 2024-04-25T01:12:03.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1463999 15.59 MB/sec execute 404 sec latency 350.641 ms 2024-04-25T01:12:04.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1464965 15.55 MB/sec execute 405 sec latency 68.515 ms 2024-04-25T01:12:05.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1467980 15.56 MB/sec execute 406 sec latency 14.732 ms 2024-04-25T01:12:06.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1471209 15.57 MB/sec execute 407 sec latency 48.712 ms 2024-04-25T01:12:07.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1474220 15.56 MB/sec execute 408 sec latency 33.672 ms 2024-04-25T01:12:08.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1476983 15.56 MB/sec execute 409 sec latency 22.968 ms 2024-04-25T01:12:09.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1479471 15.56 MB/sec execute 410 sec latency 17.385 ms 2024-04-25T01:12:10.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1482149 15.56 MB/sec execute 411 sec latency 23.893 ms 2024-04-25T01:12:11.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1484847 15.55 MB/sec execute 412 sec latency 29.989 ms 2024-04-25T01:12:12.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1486808 15.54 MB/sec execute 413 sec latency 30.182 ms 2024-04-25T01:12:13.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1489259 15.54 MB/sec execute 414 sec latency 15.594 ms 2024-04-25T01:12:14.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1491937 15.53 MB/sec execute 415 sec latency 50.379 ms 2024-04-25T01:12:15.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1493781 15.52 MB/sec execute 416 sec latency 22.097 ms 2024-04-25T01:12:16.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1496475 15.52 MB/sec execute 417 sec latency 24.417 ms 2024-04-25T01:12:17.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1499592 15.52 MB/sec execute 418 sec latency 22.727 ms 2024-04-25T01:12:18.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1502383 15.52 MB/sec execute 419 sec latency 39.902 ms 2024-04-25T01:12:19.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1504959 15.51 MB/sec execute 420 sec latency 12.246 ms 2024-04-25T01:12:20.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1507187 15.51 MB/sec execute 421 sec latency 24.378 ms 2024-04-25T01:12:21.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1510409 15.51 MB/sec execute 422 sec latency 11.447 ms 2024-04-25T01:12:22.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1513155 15.51 MB/sec execute 423 sec latency 36.686 ms 2024-04-25T01:12:23.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1514116 15.48 MB/sec execute 424 sec latency 238.032 ms 2024-04-25T01:12:24.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1515039 15.46 MB/sec execute 425 sec latency 187.170 ms 2024-04-25T01:12:25.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1517254 15.46 MB/sec execute 426 sec latency 49.875 ms 2024-04-25T01:12:26.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1520244 15.45 MB/sec execute 427 sec latency 24.955 ms 2024-04-25T01:12:27.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1522553 15.44 MB/sec execute 428 sec latency 43.056 ms 2024-04-25T01:12:28.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1525187 15.44 MB/sec execute 429 sec latency 23.344 ms 2024-04-25T01:12:29.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1528249 15.45 MB/sec execute 430 sec latency 31.627 ms 2024-04-25T01:12:30.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1531358 15.45 MB/sec execute 431 sec latency 19.610 ms 2024-04-25T01:12:31.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1534501 15.45 MB/sec execute 432 sec latency 20.115 ms 2024-04-25T01:12:32.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1537007 15.44 MB/sec execute 433 sec latency 59.147 ms 2024-04-25T01:12:33.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1539447 15.44 MB/sec execute 434 sec latency 20.540 ms 2024-04-25T01:12:34.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1542486 15.44 MB/sec execute 435 sec latency 29.252 ms 2024-04-25T01:12:35.877 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1545646 15.45 MB/sec execute 436 sec latency 28.942 ms 2024-04-25T01:12:36.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1548655 15.45 MB/sec execute 437 sec latency 29.732 ms 2024-04-25T01:12:37.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1551333 15.44 MB/sec execute 438 sec latency 29.772 ms 2024-04-25T01:12:38.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1553916 15.45 MB/sec execute 439 sec latency 21.591 ms 2024-04-25T01:12:39.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1556667 15.44 MB/sec execute 440 sec latency 27.584 ms 2024-04-25T01:12:40.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1559791 15.45 MB/sec execute 441 sec latency 27.321 ms 2024-04-25T01:12:41.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1563031 15.46 MB/sec execute 442 sec latency 39.424 ms 2024-04-25T01:12:42.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1565468 15.44 MB/sec execute 443 sec latency 55.707 ms 2024-04-25T01:12:43.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1568192 15.44 MB/sec execute 444 sec latency 13.476 ms 2024-04-25T01:12:44.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1570271 15.44 MB/sec execute 445 sec latency 249.019 ms 2024-04-25T01:12:45.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1570810 15.41 MB/sec execute 446 sec latency 264.478 ms 2024-04-25T01:12:46.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1572689 15.39 MB/sec execute 447 sec latency 61.046 ms 2024-04-25T01:12:47.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1575190 15.39 MB/sec execute 448 sec latency 57.243 ms 2024-04-25T01:12:48.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1578007 15.39 MB/sec execute 449 sec latency 20.042 ms 2024-04-25T01:12:49.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1581153 15.40 MB/sec execute 450 sec latency 33.414 ms 2024-04-25T01:12:50.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1584108 15.39 MB/sec execute 451 sec latency 20.334 ms 2024-04-25T01:12:51.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1586759 15.39 MB/sec execute 452 sec latency 48.808 ms 2024-04-25T01:12:52.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1589072 15.38 MB/sec execute 453 sec latency 14.898 ms 2024-04-25T01:12:53.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1592001 15.39 MB/sec execute 454 sec latency 22.490 ms 2024-04-25T01:12:54.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1595070 15.39 MB/sec execute 455 sec latency 42.368 ms 2024-04-25T01:12:55.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1597488 15.38 MB/sec execute 456 sec latency 22.780 ms 2024-04-25T01:12:56.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1599766 15.38 MB/sec execute 457 sec latency 17.333 ms 2024-04-25T01:12:57.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1602687 15.38 MB/sec execute 458 sec latency 31.237 ms 2024-04-25T01:12:58.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1605843 15.39 MB/sec execute 459 sec latency 35.792 ms 2024-04-25T01:12:59.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1608431 15.38 MB/sec execute 460 sec latency 25.921 ms 2024-04-25T01:13:00.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1611270 15.37 MB/sec execute 461 sec latency 35.473 ms 2024-04-25T01:13:01.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1613950 15.37 MB/sec execute 462 sec latency 13.808 ms 2024-04-25T01:13:02.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1616969 15.38 MB/sec execute 463 sec latency 47.887 ms 2024-04-25T01:13:03.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1620203 15.39 MB/sec execute 464 sec latency 23.183 ms 2024-04-25T01:13:04.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1623113 15.38 MB/sec execute 465 sec latency 27.212 ms 2024-04-25T01:13:05.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1625750 15.38 MB/sec execute 466 sec latency 36.024 ms 2024-04-25T01:13:06.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1628429 15.38 MB/sec execute 467 sec latency 15.206 ms 2024-04-25T01:13:07.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1631129 15.38 MB/sec execute 468 sec latency 50.424 ms 2024-04-25T01:13:08.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1634237 15.38 MB/sec execute 469 sec latency 30.378 ms 2024-04-25T01:13:09.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1636552 15.37 MB/sec execute 470 sec latency 43.238 ms 2024-04-25T01:13:10.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1639203 15.37 MB/sec execute 471 sec latency 19.628 ms 2024-04-25T01:13:11.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1642006 15.37 MB/sec execute 472 sec latency 21.993 ms 2024-04-25T01:13:12.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1645179 15.38 MB/sec execute 473 sec latency 34.258 ms 2024-04-25T01:13:13.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1648415 15.38 MB/sec execute 474 sec latency 18.914 ms 2024-04-25T01:13:14.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1650940 15.37 MB/sec execute 475 sec latency 32.313 ms 2024-04-25T01:13:15.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1651703 15.36 MB/sec execute 476 sec latency 776.782 ms 2024-04-25T01:13:16.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1652021 15.33 MB/sec execute 477 sec latency 1112.698 ms 2024-04-25T01:13:17.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1652458 15.30 MB/sec execute 478 sec latency 60.814 ms 2024-04-25T01:13:18.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1655342 15.30 MB/sec execute 479 sec latency 51.061 ms 2024-04-25T01:13:19.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1657864 15.29 MB/sec execute 480 sec latency 24.799 ms 2024-04-25T01:13:20.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1660504 15.29 MB/sec execute 481 sec latency 34.564 ms 2024-04-25T01:13:21.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1663291 15.29 MB/sec execute 482 sec latency 23.516 ms 2024-04-25T01:13:22.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1666434 15.30 MB/sec execute 483 sec latency 36.341 ms 2024-04-25T01:13:23.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1669610 15.30 MB/sec execute 484 sec latency 29.383 ms 2024-04-25T01:13:24.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1672386 15.30 MB/sec execute 485 sec latency 27.670 ms 2024-04-25T01:13:25.881 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1674458 15.29 MB/sec execute 486 sec latency 32.896 ms 2024-04-25T01:13:26.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1677174 15.29 MB/sec execute 487 sec latency 24.272 ms 2024-04-25T01:13:27.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1680256 15.29 MB/sec execute 488 sec latency 23.518 ms 2024-04-25T01:13:28.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1682659 15.28 MB/sec execute 489 sec latency 36.514 ms 2024-04-25T01:13:29.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1684753 15.28 MB/sec execute 490 sec latency 33.515 ms 2024-04-25T01:13:30.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1688131 15.28 MB/sec execute 491 sec latency 10.433 ms 2024-04-25T01:13:31.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1691343 15.29 MB/sec execute 492 sec latency 48.023 ms 2024-04-25T01:13:32.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1694657 15.30 MB/sec execute 493 sec latency 27.522 ms 2024-04-25T01:13:33.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1698064 15.30 MB/sec execute 494 sec latency 10.235 ms 2024-04-25T01:13:34.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1700611 15.29 MB/sec execute 495 sec latency 43.912 ms 2024-04-25T01:13:35.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1703705 15.29 MB/sec execute 496 sec latency 22.751 ms 2024-04-25T01:13:36.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1705740 15.29 MB/sec execute 497 sec latency 35.031 ms 2024-04-25T01:13:37.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1708930 15.30 MB/sec execute 498 sec latency 27.072 ms 2024-04-25T01:13:38.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1712112 15.30 MB/sec execute 499 sec latency 19.536 ms 2024-04-25T01:13:39.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1714886 15.29 MB/sec execute 500 sec latency 49.038 ms 2024-04-25T01:13:40.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1717444 15.29 MB/sec execute 501 sec latency 30.825 ms 2024-04-25T01:13:41.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1720079 15.29 MB/sec execute 502 sec latency 32.563 ms 2024-04-25T01:13:42.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1723119 15.30 MB/sec execute 503 sec latency 35.254 ms 2024-04-25T01:13:43.881 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1726327 15.30 MB/sec execute 504 sec latency 21.390 ms 2024-04-25T01:13:44.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1729064 15.29 MB/sec execute 505 sec latency 22.853 ms 2024-04-25T01:13:45.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1732058 15.29 MB/sec execute 506 sec latency 35.335 ms 2024-04-25T01:13:46.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1734274 15.29 MB/sec execute 507 sec latency 30.871 ms 2024-04-25T01:13:47.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1737490 15.30 MB/sec execute 508 sec latency 21.689 ms 2024-04-25T01:13:48.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1740656 15.30 MB/sec execute 509 sec latency 15.860 ms 2024-04-25T01:13:49.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1743747 15.30 MB/sec execute 510 sec latency 42.488 ms 2024-04-25T01:13:50.878 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1746334 15.29 MB/sec execute 511 sec latency 31.682 ms 2024-04-25T01:13:51.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1748738 15.29 MB/sec execute 512 sec latency 48.610 ms 2024-04-25T01:13:52.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1751377 15.30 MB/sec execute 513 sec latency 96.380 ms 2024-04-25T01:13:53.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1753977 15.29 MB/sec execute 514 sec latency 40.923 ms 2024-04-25T01:13:54.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1756124 15.28 MB/sec execute 515 sec latency 24.739 ms 2024-04-25T01:13:55.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1758946 15.28 MB/sec execute 516 sec latency 29.221 ms 2024-04-25T01:13:56.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1762272 15.29 MB/sec execute 517 sec latency 36.324 ms 2024-04-25T01:13:57.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1764627 15.28 MB/sec execute 518 sec latency 106.288 ms 2024-04-25T01:13:58.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1764630 15.25 MB/sec execute 519 sec latency 983.276 ms 2024-04-25T01:13:59.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1765470 15.24 MB/sec execute 520 sec latency 421.956 ms 2024-04-25T01:14:00.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1765933 15.21 MB/sec execute 521 sec latency 61.447 ms 2024-04-25T01:14:01.244 INFO:teuthology.orchestra.run.smithi171.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-mds.b.log.1: 2024-04-25T01:14:01.244 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: stdin: file size changed while zipping 2024-04-25T01:14:01.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1768763 15.21 MB/sec execute 522 sec latency 54.807 ms 2024-04-25T01:14:02.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1771253 15.20 MB/sec execute 523 sec latency 75.532 ms 2024-04-25T01:14:03.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1773268 15.19 MB/sec execute 524 sec latency 36.586 ms 2024-04-25T01:14:04.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1776488 15.20 MB/sec execute 525 sec latency 15.837 ms 2024-04-25T01:14:05.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1779720 15.21 MB/sec execute 526 sec latency 38.192 ms 2024-04-25T01:14:06.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1782627 15.20 MB/sec execute 527 sec latency 84.571 ms 2024-04-25T01:14:07.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1785551 15.20 MB/sec execute 528 sec latency 34.345 ms 2024-04-25T01:14:08.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1788172 15.20 MB/sec execute 529 sec latency 28.904 ms 2024-04-25T01:14:09.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1790895 15.20 MB/sec execute 530 sec latency 25.255 ms 2024-04-25T01:14:10.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1794235 15.21 MB/sec execute 531 sec latency 17.383 ms 2024-04-25T01:14:11.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1797404 15.22 MB/sec execute 532 sec latency 34.331 ms 2024-04-25T01:14:12.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1800332 15.21 MB/sec execute 533 sec latency 32.114 ms 2024-04-25T01:14:13.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1802599 15.21 MB/sec execute 534 sec latency 21.383 ms 2024-04-25T01:14:14.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1805060 15.20 MB/sec execute 535 sec latency 50.945 ms 2024-04-25T01:14:15.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1808124 15.21 MB/sec execute 536 sec latency 20.939 ms 2024-04-25T01:14:16.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1810695 15.20 MB/sec execute 537 sec latency 48.630 ms 2024-04-25T01:14:17.879 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1812487 15.19 MB/sec execute 538 sec latency 23.098 ms 2024-04-25T01:14:18.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1813715 15.17 MB/sec execute 539 sec latency 275.658 ms 2024-04-25T01:14:19.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1815010 15.17 MB/sec execute 540 sec latency 68.148 ms 2024-04-25T01:14:20.882 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1817885 15.16 MB/sec execute 541 sec latency 41.313 ms 2024-04-25T01:14:21.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1820874 15.16 MB/sec execute 542 sec latency 43.625 ms 2024-04-25T01:14:22.882 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1823673 15.16 MB/sec execute 543 sec latency 25.980 ms 2024-04-25T01:14:23.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1826789 15.17 MB/sec execute 544 sec latency 22.917 ms 2024-04-25T01:14:24.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1829953 15.17 MB/sec execute 545 sec latency 25.070 ms 2024-04-25T01:14:25.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1833306 15.18 MB/sec execute 546 sec latency 32.456 ms 2024-04-25T01:14:26.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1836585 15.21 MB/sec execute 547 sec latency 38.451 ms 2024-04-25T01:14:27.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1839359 15.21 MB/sec execute 548 sec latency 34.243 ms 2024-04-25T01:14:28.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1841984 15.21 MB/sec execute 549 sec latency 35.088 ms 2024-04-25T01:14:29.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1844762 15.21 MB/sec execute 550 sec latency 36.641 ms 2024-04-25T01:14:30.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1848127 15.22 MB/sec execute 551 sec latency 8.858 ms 2024-04-25T01:14:31.883 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1851387 15.22 MB/sec execute 552 sec latency 26.625 ms 2024-04-25T01:14:32.884 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1854090 15.22 MB/sec execute 553 sec latency 34.170 ms 2024-04-25T01:14:33.884 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1857069 15.22 MB/sec execute 554 sec latency 24.274 ms 2024-04-25T01:14:34.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1859675 15.22 MB/sec execute 555 sec latency 21.303 ms 2024-04-25T01:14:35.415 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:14:35.427 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:14:35.887 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1862459 15.22 MB/sec execute 556 sec latency 41.554 ms 2024-04-25T01:14:36.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1865610 15.22 MB/sec execute 557 sec latency 15.278 ms 2024-04-25T01:14:37.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1868689 15.22 MB/sec execute 558 sec latency 11.567 ms 2024-04-25T01:14:38.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1871088 15.21 MB/sec execute 559 sec latency 41.442 ms 2024-04-25T01:14:39.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1873619 15.21 MB/sec execute 560 sec latency 12.464 ms 2024-04-25T01:14:40.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1876779 15.22 MB/sec execute 561 sec latency 13.079 ms 2024-04-25T01:14:41.886 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1879812 15.22 MB/sec execute 562 sec latency 39.390 ms 2024-04-25T01:14:42.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1882695 15.22 MB/sec execute 563 sec latency 13.079 ms 2024-04-25T01:14:43.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1884675 15.21 MB/sec execute 564 sec latency 40.223 ms 2024-04-25T01:14:44.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1887525 15.21 MB/sec execute 565 sec latency 11.657 ms 2024-04-25T01:14:45.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1890735 15.22 MB/sec execute 566 sec latency 10.486 ms 2024-04-25T01:14:46.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1893504 15.22 MB/sec execute 567 sec latency 41.460 ms 2024-04-25T01:14:47.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1896147 15.21 MB/sec execute 568 sec latency 12.976 ms 2024-04-25T01:14:48.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1898169 15.21 MB/sec execute 569 sec latency 112.244 ms 2024-04-25T01:14:49.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1898182 15.18 MB/sec execute 570 sec latency 783.958 ms 2024-04-25T01:14:50.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1900573 15.18 MB/sec execute 571 sec latency 358.271 ms 2024-04-25T01:14:51.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1901105 15.16 MB/sec execute 572 sec latency 85.872 ms 2024-04-25T01:14:52.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1903150 15.14 MB/sec execute 573 sec latency 31.434 ms 2024-04-25T01:14:53.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1905411 15.14 MB/sec execute 574 sec latency 15.142 ms 2024-04-25T01:14:54.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1908327 15.15 MB/sec execute 575 sec latency 45.531 ms 2024-04-25T01:14:55.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1911256 15.14 MB/sec execute 576 sec latency 13.040 ms 2024-04-25T01:14:56.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1912746 15.13 MB/sec execute 577 sec latency 117.256 ms 2024-04-25T01:14:57.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1915744 15.13 MB/sec execute 578 sec latency 22.891 ms 2024-04-25T01:14:58.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1918989 15.14 MB/sec execute 579 sec latency 43.346 ms 2024-04-25T01:14:59.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1922334 15.15 MB/sec execute 580 sec latency 15.331 ms 2024-04-25T01:15:00.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1925373 15.15 MB/sec execute 581 sec latency 36.225 ms 2024-04-25T01:15:01.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1928054 15.14 MB/sec execute 582 sec latency 20.163 ms 2024-04-25T01:15:02.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1930617 15.14 MB/sec execute 583 sec latency 17.564 ms 2024-04-25T01:15:03.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1933470 15.14 MB/sec execute 584 sec latency 29.800 ms 2024-04-25T01:15:04.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1936525 15.15 MB/sec execute 585 sec latency 40.093 ms 2024-04-25T01:15:05.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1939306 15.14 MB/sec execute 586 sec latency 21.657 ms 2024-04-25T01:15:06.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:15:06.117 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:15:06.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1942165 15.14 MB/sec execute 587 sec latency 20.380 ms 2024-04-25T01:15:07.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1944379 15.14 MB/sec execute 588 sec latency 52.461 ms 2024-04-25T01:15:08.888 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1947464 15.14 MB/sec execute 589 sec latency 29.561 ms 2024-04-25T01:15:09.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1950542 15.15 MB/sec execute 590 sec latency 40.147 ms 2024-04-25T01:15:10.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1953368 15.14 MB/sec execute 591 sec latency 26.590 ms 2024-04-25T01:15:11.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1955899 15.14 MB/sec execute 592 sec latency 23.577 ms 2024-04-25T01:15:12.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1958537 15.14 MB/sec execute 593 sec latency 48.143 ms 2024-04-25T01:15:13.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1961813 15.14 MB/sec execute 594 sec latency 60.529 ms 2024-04-25T01:15:14.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1965085 15.15 MB/sec execute 595 sec latency 27.017 ms 2024-04-25T01:15:15.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1968446 15.16 MB/sec execute 596 sec latency 48.560 ms 2024-04-25T01:15:16.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1971368 15.15 MB/sec execute 597 sec latency 20.888 ms 2024-04-25T01:15:17.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1974121 15.15 MB/sec execute 598 sec latency 22.215 ms 2024-04-25T01:15:18.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 1976211 15.15 MB/sec execute 599 sec latency 25.318 ms 2024-04-25T01:15:19.889 INFO:tasks.workunit.client.1.smithi171.stdout: 1 cleanup 600 sec 2024-04-25T01:15:20.024 INFO:tasks.workunit.client.1.smithi171.stdout: 0 cleanup 600 sec 2024-04-25T01:15:20.024 INFO:tasks.workunit.client.1.smithi171.stdout: 2024-04-25T01:15:20.024 INFO:tasks.workunit.client.1.smithi171.stdout: Operation Count AvgLat MaxLat 2024-04-25T01:15:20.024 INFO:tasks.workunit.client.1.smithi171.stdout: ---------------------------------------- 2024-04-25T01:15:20.024 INFO:tasks.workunit.client.1.smithi171.stdout: NTCreateX 289379 0.694 332.965 2024-04-25T01:15:20.024 INFO:tasks.workunit.client.1.smithi171.stdout: Close 212555 0.002 0.101 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Rename 12250 3.353 187.159 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Unlink 58441 1.199 257.638 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Deltree 8 49.345 63.426 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Mkdir 4 0.006 0.007 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Qpathinfo 262251 0.540 642.145 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Qfileinfo 45976 0.001 0.020 2024-04-25T01:15:20.025 INFO:tasks.workunit.client.1.smithi171.stdout: Qfsinfo 48088 0.131 38.647 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: Sfileinfo 23566 0.015 8.295 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: Find 101392 0.058 34.669 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: WriteX 144376 0.017 275.650 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: ReadX 453487 0.020 1112.691 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: LockX 942 0.675 4.818 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: UnlockX 942 0.543 4.130 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: Flush 20277 5.783 983.269 2024-04-25T01:15:20.026 INFO:tasks.workunit.client.1.smithi171.stdout: 2024-04-25T01:15:20.027 INFO:tasks.workunit.client.1.smithi171.stdout:Throughput 15.1472 MB/sec 1 clients 1 procs max_latency=1112.698 ms 2024-04-25T01:15:20.027 INFO:tasks.workunit.client.1.smithi171.stderr:+ dbench 10 2024-04-25T01:15:20.047 INFO:tasks.workunit.client.1.smithi171.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T01:15:20.047 INFO:tasks.workunit.client.1.smithi171.stdout: 2024-04-25T01:15:20.047 INFO:tasks.workunit.client.1.smithi171.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T01:15:20.047 INFO:tasks.workunit.client.1.smithi171.stdout:2 of 10 processes prepared for launch 0 sec 2024-04-25T01:15:20.147 INFO:tasks.workunit.client.1.smithi171.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-25T01:15:20.147 INFO:tasks.workunit.client.1.smithi171.stdout:releasing clients 2024-04-25T01:15:21.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 430 151.78 MB/sec warmup 1 sec latency 83.188 ms 2024-04-25T01:15:22.147 INFO:tasks.workunit.client.1.smithi171.stdout: 10 929 109.24 MB/sec warmup 2 sec latency 63.239 ms 2024-04-25T01:15:23.146 INFO:tasks.workunit.client.1.smithi171.stdout: 10 1681 87.42 MB/sec warmup 3 sec latency 41.625 ms 2024-04-25T01:15:24.146 INFO:tasks.workunit.client.1.smithi171.stdout: 10 2569 72.32 MB/sec warmup 4 sec latency 36.067 ms 2024-04-25T01:15:25.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 3409 72.61 MB/sec warmup 5 sec latency 48.125 ms 2024-04-25T01:15:26.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 3785 66.26 MB/sec warmup 6 sec latency 158.283 ms 2024-04-25T01:15:27.150 INFO:tasks.workunit.client.1.smithi171.stdout: 10 3944 57.23 MB/sec warmup 7 sec latency 141.248 ms 2024-04-25T01:15:28.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 4164 50.16 MB/sec warmup 8 sec latency 92.063 ms 2024-04-25T01:15:29.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 4464 45.31 MB/sec warmup 9 sec latency 156.983 ms 2024-04-25T01:15:30.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 5185 45.28 MB/sec warmup 10 sec latency 38.571 ms 2024-04-25T01:15:31.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 5952 42.76 MB/sec warmup 11 sec latency 150.767 ms 2024-04-25T01:15:32.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 6494 43.61 MB/sec warmup 12 sec latency 63.526 ms 2024-04-25T01:15:33.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 7194 44.43 MB/sec warmup 13 sec latency 97.697 ms 2024-04-25T01:15:34.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 7454 42.21 MB/sec warmup 14 sec latency 62.884 ms 2024-04-25T01:15:35.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 7670 39.47 MB/sec warmup 15 sec latency 84.175 ms 2024-04-25T01:15:36.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 7729 37.04 MB/sec warmup 16 sec latency 590.493 ms 2024-04-25T01:15:36.176 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:15:36.179 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:15:37.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 7782 34.91 MB/sec warmup 17 sec latency 935.732 ms 2024-04-25T01:15:38.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 7901 33.14 MB/sec warmup 18 sec latency 191.618 ms 2024-04-25T01:15:39.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 8247 32.22 MB/sec warmup 19 sec latency 123.590 ms 2024-04-25T01:15:40.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 8942 32.29 MB/sec warmup 20 sec latency 44.367 ms 2024-04-25T01:15:41.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 9726 32.29 MB/sec warmup 21 sec latency 30.976 ms 2024-04-25T01:15:42.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 10543 34.16 MB/sec warmup 22 sec latency 72.189 ms 2024-04-25T01:15:43.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 10911 33.91 MB/sec warmup 23 sec latency 61.651 ms 2024-04-25T01:15:44.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 11116 32.63 MB/sec warmup 24 sec latency 94.760 ms 2024-04-25T01:15:45.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 11325 31.40 MB/sec warmup 25 sec latency 76.262 ms 2024-04-25T01:15:46.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 11428 30.29 MB/sec warmup 26 sec latency 516.445 ms 2024-04-25T01:15:47.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 11434 29.17 MB/sec warmup 27 sec latency 1464.566 ms 2024-04-25T01:15:48.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 11465 28.15 MB/sec warmup 28 sec latency 1799.774 ms 2024-04-25T01:15:49.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 11704 27.53 MB/sec warmup 29 sec latency 126.963 ms 2024-04-25T01:15:50.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 12095 27.51 MB/sec warmup 30 sec latency 89.966 ms 2024-04-25T01:15:51.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 12959 27.44 MB/sec warmup 31 sec latency 52.089 ms 2024-04-25T01:15:52.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 13704 28.45 MB/sec warmup 32 sec latency 32.634 ms 2024-04-25T01:15:53.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 14266 28.97 MB/sec warmup 33 sec latency 80.201 ms 2024-04-25T01:15:54.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 14534 28.62 MB/sec warmup 34 sec latency 80.421 ms 2024-04-25T01:15:55.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 14744 27.86 MB/sec warmup 35 sec latency 67.757 ms 2024-04-25T01:15:56.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 14978 27.19 MB/sec warmup 36 sec latency 156.538 ms 2024-04-25T01:15:57.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 15442 27.10 MB/sec warmup 37 sec latency 114.141 ms 2024-04-25T01:15:58.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 16378 27.24 MB/sec warmup 38 sec latency 50.053 ms 2024-04-25T01:15:59.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 17082 27.98 MB/sec warmup 39 sec latency 34.919 ms 2024-04-25T01:16:00.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 17741 28.69 MB/sec warmup 40 sec latency 63.845 ms 2024-04-25T01:16:01.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 18025 28.36 MB/sec warmup 41 sec latency 163.139 ms 2024-04-25T01:16:02.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 18036 27.69 MB/sec warmup 42 sec latency 638.947 ms 2024-04-25T01:16:03.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 18097 27.08 MB/sec warmup 43 sec latency 755.389 ms 2024-04-25T01:16:04.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 18261 26.50 MB/sec warmup 44 sec latency 99.415 ms 2024-04-25T01:16:05.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 18447 25.97 MB/sec warmup 45 sec latency 157.260 ms 2024-04-25T01:16:06.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 18752 25.63 MB/sec warmup 46 sec latency 102.473 ms 2024-04-25T01:16:06.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:16:06.494 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:16:06.525 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:16:06.540+0000 7f224640f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41730) UID: 0 2024-04-25T01:16:06.526 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:16:06.540+0000 7ff199406640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41730) UID: 0 2024-04-25T01:16:07.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 19399 25.94 MB/sec warmup 47 sec latency 44.024 ms 2024-04-25T01:16:08.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 20280 25.91 MB/sec warmup 48 sec latency 46.151 ms 2024-04-25T01:16:09.148 INFO:tasks.workunit.client.1.smithi171.stdout: 10 20860 26.49 MB/sec warmup 49 sec latency 70.031 ms 2024-04-25T01:16:10.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 21424 26.87 MB/sec warmup 50 sec latency 90.394 ms 2024-04-25T01:16:11.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 21677 26.58 MB/sec warmup 51 sec latency 86.844 ms 2024-04-25T01:16:12.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 21861 26.11 MB/sec warmup 52 sec latency 152.370 ms 2024-04-25T01:16:13.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 22065 25.68 MB/sec warmup 53 sec latency 123.583 ms 2024-04-25T01:16:14.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 22433 25.50 MB/sec warmup 54 sec latency 92.399 ms 2024-04-25T01:16:15.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 23062 25.67 MB/sec warmup 55 sec latency 41.555 ms 2024-04-25T01:16:16.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 23822 25.62 MB/sec warmup 56 sec latency 44.347 ms 2024-04-25T01:16:17.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 24346 26.08 MB/sec warmup 57 sec latency 107.265 ms 2024-04-25T01:16:18.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 24920 26.44 MB/sec warmup 58 sec latency 78.028 ms 2024-04-25T01:16:19.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 25177 26.21 MB/sec warmup 59 sec latency 103.198 ms 2024-04-25T01:16:20.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 25356 25.81 MB/sec warmup 60 sec latency 82.310 ms 2024-04-25T01:16:21.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 25549 25.43 MB/sec warmup 61 sec latency 166.928 ms 2024-04-25T01:16:22.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 25813 25.16 MB/sec warmup 62 sec latency 143.678 ms 2024-04-25T01:16:23.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 26348 25.23 MB/sec warmup 63 sec latency 60.832 ms 2024-04-25T01:16:24.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 27126 25.23 MB/sec warmup 64 sec latency 50.701 ms 2024-04-25T01:16:25.165 INFO:tasks.workunit.client.1.smithi171.stdout: 10 27782 25.60 MB/sec warmup 65 sec latency 37.937 ms 2024-04-25T01:16:26.150 INFO:tasks.workunit.client.1.smithi171.stdout: 10 28358 25.94 MB/sec warmup 66 sec latency 78.607 ms 2024-04-25T01:16:27.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 28647 25.88 MB/sec warmup 67 sec latency 114.818 ms 2024-04-25T01:16:28.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 28852 25.58 MB/sec warmup 68 sec latency 109.390 ms 2024-04-25T01:16:29.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 29030 25.23 MB/sec warmup 69 sec latency 152.346 ms 2024-04-25T01:16:30.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 29319 25.02 MB/sec warmup 70 sec latency 93.824 ms 2024-04-25T01:16:31.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 29862 25.06 MB/sec warmup 71 sec latency 49.795 ms 2024-04-25T01:16:32.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 30598 25.11 MB/sec warmup 72 sec latency 95.276 ms 2024-04-25T01:16:33.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 30690 24.80 MB/sec warmup 73 sec latency 931.782 ms 2024-04-25T01:16:34.149 INFO:tasks.workunit.client.1.smithi171.stdout: 10 30696 24.47 MB/sec warmup 74 sec latency 1875.864 ms 2024-04-25T01:16:35.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 30735 24.15 MB/sec warmup 75 sec latency 2257.535 ms 2024-04-25T01:16:36.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 31060 24.09 MB/sec warmup 76 sec latency 232.801 ms 2024-04-25T01:16:37.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 31285 24.04 MB/sec warmup 77 sec latency 155.647 ms 2024-04-25T01:16:38.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 31636 24.12 MB/sec warmup 78 sec latency 133.767 ms 2024-04-25T01:16:39.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 32079 24.25 MB/sec warmup 79 sec latency 71.161 ms 2024-04-25T01:16:40.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 32316 24.12 MB/sec warmup 80 sec latency 89.050 ms 2024-04-25T01:16:41.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 32489 23.84 MB/sec warmup 81 sec latency 174.668 ms 2024-04-25T01:16:42.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 32704 23.60 MB/sec warmup 82 sec latency 155.982 ms 2024-04-25T01:16:43.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 33033 23.49 MB/sec warmup 83 sec latency 133.626 ms 2024-04-25T01:16:44.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 33615 23.53 MB/sec warmup 84 sec latency 86.329 ms 2024-04-25T01:16:45.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 34328 23.60 MB/sec warmup 85 sec latency 53.091 ms 2024-04-25T01:16:46.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 34935 23.85 MB/sec warmup 86 sec latency 43.128 ms 2024-04-25T01:16:47.151 INFO:tasks.workunit.client.1.smithi171.stdout: 10 35356 23.98 MB/sec warmup 87 sec latency 118.374 ms 2024-04-25T01:16:48.153 INFO:tasks.workunit.client.1.smithi171.stdout: 10 35591 23.92 MB/sec warmup 88 sec latency 526.317 ms 2024-04-25T01:16:49.152 INFO:tasks.workunit.client.1.smithi171.stdout: 10 35643 23.70 MB/sec warmup 89 sec latency 1222.952 ms 2024-04-25T01:16:50.152 INFO:tasks.workunit.client.1.smithi171.stdout: 10 35841 23.56 MB/sec warmup 90 sec latency 89.214 ms 2024-04-25T01:16:51.152 INFO:tasks.workunit.client.1.smithi171.stdout: 10 36019 23.32 MB/sec warmup 91 sec latency 122.185 ms 2024-04-25T01:16:52.155 INFO:tasks.workunit.client.1.smithi171.stdout: 10 36233 23.11 MB/sec warmup 92 sec latency 98.670 ms 2024-04-25T01:16:53.152 INFO:tasks.workunit.client.1.smithi171.stdout: 10 36526 23.01 MB/sec warmup 93 sec latency 112.705 ms 2024-04-25T01:16:54.153 INFO:tasks.workunit.client.1.smithi171.stdout: 10 37056 23.00 MB/sec warmup 94 sec latency 82.942 ms 2024-04-25T01:16:55.152 INFO:tasks.workunit.client.1.smithi171.stdout: 10 37821 23.15 MB/sec warmup 95 sec latency 43.594 ms 2024-04-25T01:16:56.155 INFO:tasks.workunit.client.1.smithi171.stdout: 10 38524 23.38 MB/sec warmup 96 sec latency 36.916 ms 2024-04-25T01:16:57.153 INFO:tasks.workunit.client.1.smithi171.stdout: 10 38983 23.56 MB/sec warmup 97 sec latency 96.847 ms 2024-04-25T01:16:58.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 39315 23.54 MB/sec warmup 98 sec latency 77.643 ms 2024-04-25T01:16:59.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 39519 23.36 MB/sec warmup 99 sec latency 113.367 ms 2024-04-25T01:17:00.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 39759 23.17 MB/sec warmup 100 sec latency 109.132 ms 2024-04-25T01:17:01.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 40078 23.08 MB/sec warmup 101 sec latency 140.828 ms 2024-04-25T01:17:02.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 40618 23.08 MB/sec warmup 102 sec latency 100.155 ms 2024-04-25T01:17:03.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 41120 23.08 MB/sec warmup 103 sec latency 215.017 ms 2024-04-25T01:17:04.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 41137 22.87 MB/sec warmup 104 sec latency 951.473 ms 2024-04-25T01:17:05.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 41754 22.96 MB/sec warmup 105 sec latency 483.258 ms 2024-04-25T01:17:06.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 42066 22.96 MB/sec warmup 106 sec latency 88.952 ms 2024-04-25T01:17:07.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 42434 23.07 MB/sec warmup 107 sec latency 161.146 ms 2024-04-25T01:17:08.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 42808 23.11 MB/sec warmup 108 sec latency 94.094 ms 2024-04-25T01:17:09.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 43034 22.97 MB/sec warmup 109 sec latency 87.493 ms 2024-04-25T01:17:10.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 43225 22.79 MB/sec warmup 110 sec latency 119.179 ms 2024-04-25T01:17:11.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 43461 22.64 MB/sec warmup 111 sec latency 126.350 ms 2024-04-25T01:17:12.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 43862 22.63 MB/sec warmup 112 sec latency 130.726 ms 2024-04-25T01:17:13.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 44554 22.66 MB/sec warmup 113 sec latency 48.672 ms 2024-04-25T01:17:14.154 INFO:tasks.workunit.client.1.smithi171.stdout: 10 45320 22.84 MB/sec warmup 114 sec latency 43.788 ms 2024-04-25T01:17:15.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 45857 23.04 MB/sec warmup 115 sec latency 98.129 ms 2024-04-25T01:17:16.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 46250 23.08 MB/sec warmup 116 sec latency 102.789 ms 2024-04-25T01:17:17.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 46511 23.02 MB/sec warmup 117 sec latency 112.978 ms 2024-04-25T01:17:18.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 46704 22.85 MB/sec warmup 118 sec latency 134.371 ms 2024-04-25T01:17:19.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 46745 22.67 MB/sec warmup 119 sec latency 814.389 ms 2024-04-25T01:17:21.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 47044 7.92 MB/sec execute 1 sec latency 1143.793 ms 2024-04-25T01:17:22.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 47229 7.06 MB/sec execute 2 sec latency 136.492 ms 2024-04-25T01:17:23.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 47762 12.13 MB/sec execute 3 sec latency 83.077 ms 2024-04-25T01:17:24.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 48474 17.99 MB/sec execute 4 sec latency 45.974 ms 2024-04-25T01:17:25.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 49181 23.93 MB/sec execute 5 sec latency 35.091 ms 2024-04-25T01:17:26.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 49691 26.63 MB/sec execute 6 sec latency 59.042 ms 2024-04-25T01:17:27.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 49948 25.66 MB/sec execute 7 sec latency 136.953 ms 2024-04-25T01:17:28.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 50140 23.15 MB/sec execute 8 sec latency 130.231 ms 2024-04-25T01:17:29.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 50350 21.02 MB/sec execute 9 sec latency 124.734 ms 2024-04-25T01:17:30.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 50640 20.08 MB/sec execute 10 sec latency 140.668 ms 2024-04-25T01:17:31.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 51167 20.35 MB/sec execute 11 sec latency 74.240 ms 2024-04-25T01:17:32.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 51819 20.99 MB/sec execute 12 sec latency 121.960 ms 2024-04-25T01:17:33.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 52471 22.54 MB/sec execute 13 sec latency 66.831 ms 2024-04-25T01:17:34.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 52816 22.73 MB/sec execute 14 sec latency 513.846 ms 2024-04-25T01:17:35.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 52843 21.29 MB/sec execute 15 sec latency 1305.860 ms 2024-04-25T01:17:36.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 53140 21.61 MB/sec execute 16 sec latency 792.243 ms 2024-04-25T01:17:37.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 53307 21.07 MB/sec execute 17 sec latency 143.363 ms 2024-04-25T01:17:38.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 53599 20.72 MB/sec execute 18 sec latency 66.848 ms 2024-04-25T01:17:39.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 53814 19.88 MB/sec execute 19 sec latency 110.556 ms 2024-04-25T01:17:40.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 54072 19.34 MB/sec execute 20 sec latency 166.116 ms 2024-04-25T01:17:41.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 54378 19.05 MB/sec execute 21 sec latency 215.849 ms 2024-04-25T01:17:42.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 54868 19.00 MB/sec execute 22 sec latency 92.860 ms 2024-04-25T01:17:43.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 55513 19.66 MB/sec execute 23 sec latency 66.511 ms 2024-04-25T01:17:44.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 56225 20.76 MB/sec execute 24 sec latency 77.912 ms 2024-04-25T01:17:45.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 56723 21.54 MB/sec execute 25 sec latency 64.287 ms 2024-04-25T01:17:46.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 57080 21.69 MB/sec execute 26 sec latency 80.671 ms 2024-04-25T01:17:47.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 57255 21.02 MB/sec execute 27 sec latency 164.870 ms 2024-04-25T01:17:48.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 57536 20.58 MB/sec execute 28 sec latency 79.992 ms 2024-04-25T01:17:49.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 57872 20.26 MB/sec execute 29 sec latency 83.485 ms 2024-04-25T01:17:50.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 58407 20.36 MB/sec execute 30 sec latency 90.671 ms 2024-04-25T01:17:51.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 59176 20.90 MB/sec execute 31 sec latency 50.435 ms 2024-04-25T01:17:52.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 59852 21.71 MB/sec execute 32 sec latency 38.310 ms 2024-04-25T01:17:53.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 60334 22.23 MB/sec execute 33 sec latency 105.103 ms 2024-04-25T01:17:54.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 60634 22.20 MB/sec execute 34 sec latency 77.859 ms 2024-04-25T01:17:55.157 INFO:tasks.workunit.client.1.smithi171.stdout: 10 60877 21.75 MB/sec execute 35 sec latency 85.356 ms 2024-04-25T01:17:56.158 INFO:tasks.workunit.client.1.smithi171.stdout: 10 61120 21.33 MB/sec execute 36 sec latency 120.461 ms 2024-04-25T01:17:57.158 INFO:tasks.workunit.client.1.smithi171.stdout: 10 61440 21.03 MB/sec execute 37 sec latency 126.221 ms 2024-04-25T01:17:58.158 INFO:tasks.workunit.client.1.smithi171.stdout: 10 61976 21.19 MB/sec execute 38 sec latency 88.392 ms 2024-04-25T01:17:59.158 INFO:tasks.workunit.client.1.smithi171.stdout: 10 62698 21.48 MB/sec execute 39 sec latency 44.234 ms 2024-04-25T01:18:00.158 INFO:tasks.workunit.client.1.smithi171.stdout: 10 62818 21.10 MB/sec execute 40 sec latency 900.565 ms 2024-04-25T01:18:01.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 62840 20.62 MB/sec execute 41 sec latency 1901.730 ms 2024-04-25T01:18:02.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 62963 20.35 MB/sec execute 42 sec latency 2058.225 ms 2024-04-25T01:18:03.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 63210 20.24 MB/sec execute 43 sec latency 108.645 ms 2024-04-25T01:18:04.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 63521 20.46 MB/sec execute 44 sec latency 98.315 ms 2024-04-25T01:18:05.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 63952 20.71 MB/sec execute 45 sec latency 84.635 ms 2024-04-25T01:18:06.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 64235 20.60 MB/sec execute 46 sec latency 76.749 ms 2024-04-25T01:18:07.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 64489 20.28 MB/sec execute 47 sec latency 87.831 ms 2024-04-25T01:18:08.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 64763 20.07 MB/sec execute 48 sec latency 146.863 ms 2024-04-25T01:18:09.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 65189 20.01 MB/sec execute 49 sec latency 97.923 ms 2024-04-25T01:18:10.162 INFO:tasks.workunit.client.1.smithi171.stdout: 10 65552 19.99 MB/sec execute 50 sec latency 299.206 ms 2024-04-25T01:18:11.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 66089 20.01 MB/sec execute 51 sec latency 676.728 ms 2024-04-25T01:18:12.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 66674 20.45 MB/sec execute 52 sec latency 82.088 ms 2024-04-25T01:18:13.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 67194 20.74 MB/sec execute 53 sec latency 94.899 ms 2024-04-25T01:18:14.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 67574 20.92 MB/sec execute 54 sec latency 91.516 ms 2024-04-25T01:18:15.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 67694 20.60 MB/sec execute 55 sec latency 575.824 ms 2024-04-25T01:18:16.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 67757 20.28 MB/sec execute 56 sec latency 1342.829 ms 2024-04-25T01:18:17.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 67903 20.01 MB/sec execute 57 sec latency 125.088 ms 2024-04-25T01:18:18.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 68109 19.75 MB/sec execute 58 sec latency 142.500 ms 2024-04-25T01:18:19.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 68414 19.65 MB/sec execute 59 sec latency 135.602 ms 2024-04-25T01:18:20.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 68843 19.61 MB/sec execute 60 sec latency 130.473 ms 2024-04-25T01:18:21.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 69419 19.68 MB/sec execute 61 sec latency 125.319 ms 2024-04-25T01:18:22.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 70052 20.00 MB/sec execute 62 sec latency 56.039 ms 2024-04-25T01:18:23.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 70651 20.35 MB/sec execute 63 sec latency 54.324 ms 2024-04-25T01:18:24.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 71023 20.52 MB/sec execute 64 sec latency 131.609 ms 2024-04-25T01:18:25.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 71313 20.46 MB/sec execute 65 sec latency 95.709 ms 2024-04-25T01:18:26.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 71523 20.21 MB/sec execute 66 sec latency 111.653 ms 2024-04-25T01:18:27.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 71775 20.05 MB/sec execute 67 sec latency 113.162 ms 2024-04-25T01:18:28.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 72181 20.01 MB/sec execute 68 sec latency 102.721 ms 2024-04-25T01:18:29.159 INFO:tasks.workunit.client.1.smithi171.stdout: 10 72782 20.07 MB/sec execute 69 sec latency 85.476 ms 2024-04-25T01:18:30.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 73505 20.31 MB/sec execute 70 sec latency 43.282 ms 2024-04-25T01:18:31.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 74199 20.74 MB/sec execute 71 sec latency 42.582 ms 2024-04-25T01:18:32.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 74606 20.92 MB/sec execute 72 sec latency 75.771 ms 2024-04-25T01:18:33.162 INFO:tasks.workunit.client.1.smithi171.stdout: 10 74901 20.84 MB/sec execute 73 sec latency 50.089 ms 2024-04-25T01:18:34.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 75102 20.62 MB/sec execute 74 sec latency 137.218 ms 2024-04-25T01:18:35.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 75370 20.48 MB/sec execute 75 sec latency 83.118 ms 2024-04-25T01:18:36.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 75405 20.22 MB/sec execute 76 sec latency 892.910 ms 2024-04-25T01:18:37.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 75441 19.96 MB/sec execute 77 sec latency 825.633 ms 2024-04-25T01:18:38.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 75499 19.72 MB/sec execute 78 sec latency 488.897 ms 2024-04-25T01:18:39.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 75663 19.61 MB/sec execute 79 sec latency 188.516 ms 2024-04-25T01:18:40.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 76220 19.58 MB/sec execute 80 sec latency 81.056 ms 2024-04-25T01:18:41.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 76902 19.72 MB/sec execute 81 sec latency 44.615 ms 2024-04-25T01:18:42.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 77510 20.00 MB/sec execute 82 sec latency 50.569 ms 2024-04-25T01:18:43.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 77929 20.21 MB/sec execute 83 sec latency 141.398 ms 2024-04-25T01:18:44.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 78340 20.35 MB/sec execute 84 sec latency 72.457 ms 2024-04-25T01:18:45.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 78543 20.18 MB/sec execute 85 sec latency 112.604 ms 2024-04-25T01:18:46.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 78753 19.99 MB/sec execute 86 sec latency 126.100 ms 2024-04-25T01:18:47.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 79093 19.89 MB/sec execute 87 sec latency 116.506 ms 2024-04-25T01:18:48.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 79688 19.96 MB/sec execute 88 sec latency 81.331 ms 2024-04-25T01:18:49.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 80527 20.19 MB/sec execute 89 sec latency 38.480 ms 2024-04-25T01:18:50.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 81217 20.56 MB/sec execute 90 sec latency 61.888 ms 2024-04-25T01:18:51.162 INFO:tasks.workunit.client.1.smithi171.stdout: 10 81435 20.53 MB/sec execute 91 sec latency 688.705 ms 2024-04-25T01:18:52.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 81511 20.34 MB/sec execute 92 sec latency 1207.387 ms 2024-04-25T01:18:53.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 81747 20.33 MB/sec execute 93 sec latency 85.968 ms 2024-04-25T01:18:54.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 82022 20.27 MB/sec execute 94 sec latency 86.656 ms 2024-04-25T01:18:55.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 82215 20.09 MB/sec execute 95 sec latency 90.033 ms 2024-04-25T01:18:56.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 82467 19.96 MB/sec execute 96 sec latency 122.837 ms 2024-04-25T01:18:57.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 82909 19.98 MB/sec execute 97 sec latency 88.019 ms 2024-04-25T01:18:58.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 83647 20.06 MB/sec execute 98 sec latency 61.041 ms 2024-04-25T01:18:59.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 84430 20.35 MB/sec execute 99 sec latency 37.473 ms 2024-04-25T01:19:00.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 85022 20.64 MB/sec execute 100 sec latency 83.600 ms 2024-04-25T01:19:01.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 85424 20.69 MB/sec execute 101 sec latency 67.304 ms 2024-04-25T01:19:02.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 85637 20.56 MB/sec execute 102 sec latency 90.508 ms 2024-04-25T01:19:03.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 85883 20.41 MB/sec execute 103 sec latency 87.029 ms 2024-04-25T01:19:04.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 86286 20.39 MB/sec execute 104 sec latency 80.886 ms 2024-04-25T01:19:05.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 86913 20.43 MB/sec execute 105 sec latency 87.010 ms 2024-04-25T01:19:06.160 INFO:tasks.workunit.client.1.smithi171.stdout: 10 87639 20.59 MB/sec execute 106 sec latency 38.745 ms 2024-04-25T01:19:07.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 88330 20.84 MB/sec execute 107 sec latency 67.820 ms 2024-04-25T01:19:08.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 88788 21.01 MB/sec execute 108 sec latency 91.976 ms 2024-04-25T01:19:09.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 89069 20.97 MB/sec execute 109 sec latency 79.024 ms 2024-04-25T01:19:10.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 89287 20.82 MB/sec execute 110 sec latency 115.890 ms 2024-04-25T01:19:11.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 89577 20.72 MB/sec execute 111 sec latency 124.778 ms 2024-04-25T01:19:12.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 90061 20.74 MB/sec execute 112 sec latency 84.289 ms 2024-04-25T01:19:13.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 90785 20.82 MB/sec execute 113 sec latency 43.964 ms 2024-04-25T01:19:13.326 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:19:13.329 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:19:13.360 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:19:13.378+0000 7f224640f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41756) UID: 0 2024-04-25T01:19:13.361 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:19:13.378+0000 7ff199406640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41756) UID: 0 2024-04-25T01:19:14.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 91432 20.97 MB/sec execute 114 sec latency 80.148 ms 2024-04-25T01:19:15.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 91956 21.13 MB/sec execute 115 sec latency 108.320 ms 2024-04-25T01:19:16.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92338 21.22 MB/sec execute 116 sec latency 121.584 ms 2024-04-25T01:19:17.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92371 21.05 MB/sec execute 117 sec latency 628.445 ms 2024-04-25T01:19:18.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92379 20.88 MB/sec execute 118 sec latency 922.619 ms 2024-04-25T01:19:19.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92423 20.72 MB/sec execute 119 sec latency 564.771 ms 2024-04-25T01:19:20.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92575 20.64 MB/sec execute 120 sec latency 159.608 ms 2024-04-25T01:19:21.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92679 20.48 MB/sec execute 121 sec latency 101.788 ms 2024-04-25T01:19:22.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92812 20.34 MB/sec execute 122 sec latency 160.174 ms 2024-04-25T01:19:23.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 92989 20.21 MB/sec execute 123 sec latency 128.826 ms 2024-04-25T01:19:24.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 93310 20.17 MB/sec execute 124 sec latency 202.400 ms 2024-04-25T01:19:25.161 INFO:tasks.workunit.client.1.smithi171.stdout: 10 93933 20.20 MB/sec execute 125 sec latency 110.957 ms 2024-04-25T01:19:26.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 94688 20.34 MB/sec execute 126 sec latency 40.445 ms 2024-04-25T01:19:27.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 95246 20.48 MB/sec execute 127 sec latency 96.904 ms 2024-04-25T01:19:28.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 95721 20.63 MB/sec execute 128 sec latency 98.152 ms 2024-04-25T01:19:29.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 96069 20.68 MB/sec execute 129 sec latency 95.027 ms 2024-04-25T01:19:30.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 96283 20.55 MB/sec execute 130 sec latency 102.283 ms 2024-04-25T01:19:31.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 96483 20.42 MB/sec execute 131 sec latency 104.754 ms 2024-04-25T01:19:32.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 96785 20.33 MB/sec execute 132 sec latency 92.724 ms 2024-04-25T01:19:33.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 97243 20.37 MB/sec execute 133 sec latency 113.064 ms 2024-04-25T01:19:34.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 97970 20.43 MB/sec execute 134 sec latency 46.368 ms 2024-04-25T01:19:35.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 98706 20.61 MB/sec execute 135 sec latency 70.706 ms 2024-04-25T01:19:36.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 99262 20.82 MB/sec execute 136 sec latency 69.880 ms 2024-04-25T01:19:37.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 99593 20.84 MB/sec execute 137 sec latency 127.508 ms 2024-04-25T01:19:38.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 99807 20.74 MB/sec execute 138 sec latency 105.964 ms 2024-04-25T01:19:39.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 100003 20.61 MB/sec execute 139 sec latency 105.960 ms 2024-04-25T01:19:40.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 100267 20.53 MB/sec execute 140 sec latency 104.281 ms 2024-04-25T01:19:41.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 100731 20.56 MB/sec execute 141 sec latency 73.638 ms 2024-04-25T01:19:42.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 101528 20.60 MB/sec execute 142 sec latency 46.014 ms 2024-04-25T01:19:43.163 INFO:tasks.workunit.client.1.smithi171.stdout: 10 102177 20.81 MB/sec execute 143 sec latency 40.568 ms 2024-04-25T01:19:44.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 102763 20.97 MB/sec execute 144 sec latency 89.452 ms 2024-04-25T01:19:45.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103091 20.99 MB/sec execute 145 sec latency 103.569 ms 2024-04-25T01:19:46.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103354 20.91 MB/sec execute 146 sec latency 113.931 ms 2024-04-25T01:19:47.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103456 20.77 MB/sec execute 147 sec latency 368.080 ms 2024-04-25T01:19:48.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103463 20.63 MB/sec execute 148 sec latency 1337.756 ms 2024-04-25T01:19:49.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103469 20.49 MB/sec execute 149 sec latency 2286.310 ms 2024-04-25T01:19:50.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103589 20.38 MB/sec execute 150 sec latency 2114.396 ms 2024-04-25T01:19:51.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103719 20.27 MB/sec execute 151 sec latency 160.141 ms 2024-04-25T01:19:52.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 103939 20.20 MB/sec execute 152 sec latency 95.276 ms 2024-04-25T01:19:53.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 104232 20.21 MB/sec execute 153 sec latency 92.388 ms 2024-04-25T01:19:54.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 104588 20.13 MB/sec execute 154 sec latency 100.650 ms 2024-04-25T01:19:55.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 105211 20.17 MB/sec execute 155 sec latency 103.719 ms 2024-04-25T01:19:56.168 INFO:tasks.workunit.client.1.smithi171.stdout: 10 105910 20.36 MB/sec execute 156 sec latency 82.959 ms 2024-04-25T01:19:57.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 106408 20.50 MB/sec execute 157 sec latency 83.074 ms 2024-04-25T01:19:58.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 106701 20.49 MB/sec execute 158 sec latency 103.233 ms 2024-04-25T01:19:59.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 106955 20.40 MB/sec execute 159 sec latency 90.728 ms 2024-04-25T01:20:00.173 INFO:tasks.workunit.client.1.smithi171.stdout: 10 107122 20.30 MB/sec execute 160 sec latency 248.487 ms 2024-04-25T01:20:01.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 107466 20.27 MB/sec execute 161 sec latency 94.184 ms 2024-04-25T01:20:02.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 107975 20.30 MB/sec execute 162 sec latency 85.003 ms 2024-04-25T01:20:03.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 108702 20.34 MB/sec execute 163 sec latency 40.643 ms 2024-04-25T01:20:04.169 INFO:tasks.workunit.client.1.smithi171.stdout: 10 109382 20.52 MB/sec execute 164 sec latency 41.672 ms 2024-04-25T01:20:05.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 109962 20.67 MB/sec execute 165 sec latency 49.286 ms 2024-04-25T01:20:06.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110211 20.64 MB/sec execute 166 sec latency 83.784 ms 2024-04-25T01:20:07.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110442 20.57 MB/sec execute 167 sec latency 94.895 ms 2024-04-25T01:20:08.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110486 20.45 MB/sec execute 168 sec latency 944.157 ms 2024-04-25T01:20:09.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110508 20.33 MB/sec execute 169 sec latency 1544.307 ms 2024-04-25T01:20:10.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110614 20.22 MB/sec execute 170 sec latency 780.774 ms 2024-04-25T01:20:11.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110722 20.12 MB/sec execute 171 sec latency 283.827 ms 2024-04-25T01:20:12.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 110880 20.04 MB/sec execute 172 sec latency 226.443 ms 2024-04-25T01:20:13.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 111322 20.09 MB/sec execute 173 sec latency 250.119 ms 2024-04-25T01:20:14.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 111992 20.07 MB/sec execute 174 sec latency 93.601 ms 2024-04-25T01:20:15.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 112525 20.13 MB/sec execute 175 sec latency 85.309 ms 2024-04-25T01:20:16.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 113244 20.34 MB/sec execute 176 sec latency 35.375 ms 2024-04-25T01:20:17.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 113604 20.38 MB/sec execute 177 sec latency 144.867 ms 2024-04-25T01:20:18.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 113883 20.35 MB/sec execute 178 sec latency 94.233 ms 2024-04-25T01:20:19.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 114096 20.27 MB/sec execute 179 sec latency 102.503 ms 2024-04-25T01:20:20.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 114340 20.19 MB/sec execute 180 sec latency 111.979 ms 2024-04-25T01:20:21.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 114826 20.23 MB/sec execute 181 sec latency 82.577 ms 2024-04-25T01:20:22.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 115516 20.23 MB/sec execute 182 sec latency 61.343 ms 2024-04-25T01:20:23.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 116015 20.30 MB/sec execute 183 sec latency 102.629 ms 2024-04-25T01:20:24.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 116026 20.20 MB/sec execute 184 sec latency 1052.420 ms 2024-04-25T01:20:25.170 INFO:tasks.workunit.client.1.smithi171.stdout: 10 116265 20.19 MB/sec execute 185 sec latency 1795.608 ms 2024-04-25T01:20:26.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 116470 20.13 MB/sec execute 186 sec latency 173.145 ms 2024-04-25T01:20:27.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 116731 20.15 MB/sec execute 187 sec latency 338.226 ms 2024-04-25T01:20:28.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 117067 20.17 MB/sec execute 188 sec latency 115.182 ms 2024-04-25T01:20:29.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 117348 20.16 MB/sec execute 189 sec latency 102.837 ms 2024-04-25T01:20:30.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 117565 20.09 MB/sec execute 190 sec latency 123.720 ms 2024-04-25T01:20:31.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 117735 20.01 MB/sec execute 191 sec latency 161.313 ms 2024-04-25T01:20:32.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 117973 19.94 MB/sec execute 192 sec latency 152.622 ms 2024-04-25T01:20:33.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 118531 19.99 MB/sec execute 193 sec latency 55.947 ms 2024-04-25T01:20:34.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 119182 19.99 MB/sec execute 194 sec latency 128.934 ms 2024-04-25T01:20:35.171 INFO:tasks.workunit.client.1.smithi171.stdout: 10 119769 20.13 MB/sec execute 195 sec latency 41.765 ms 2024-04-25T01:20:36.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 120384 20.25 MB/sec execute 196 sec latency 94.020 ms 2024-04-25T01:20:37.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 120677 20.26 MB/sec execute 197 sec latency 183.876 ms 2024-04-25T01:20:38.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 120974 20.21 MB/sec execute 198 sec latency 63.462 ms 2024-04-25T01:20:39.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 121097 20.14 MB/sec execute 199 sec latency 636.687 ms 2024-04-25T01:20:40.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 121123 20.04 MB/sec execute 200 sec latency 950.911 ms 2024-04-25T01:20:41.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 121234 19.95 MB/sec execute 201 sec latency 180.555 ms 2024-04-25T01:20:42.176 INFO:tasks.workunit.client.1.smithi171.stdout: 10 121390 19.88 MB/sec execute 202 sec latency 151.232 ms 2024-04-25T01:20:43.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 121849 19.91 MB/sec execute 203 sec latency 127.495 ms 2024-04-25T01:20:44.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 122367 19.88 MB/sec execute 204 sec latency 121.655 ms 2024-04-25T01:20:45.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 122980 19.93 MB/sec execute 205 sec latency 44.490 ms 2024-04-25T01:20:46.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 123592 20.06 MB/sec execute 206 sec latency 78.709 ms 2024-04-25T01:20:47.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 123989 20.11 MB/sec execute 207 sec latency 248.874 ms 2024-04-25T01:20:48.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 124292 20.12 MB/sec execute 208 sec latency 107.128 ms 2024-04-25T01:20:49.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 124519 20.07 MB/sec execute 209 sec latency 119.937 ms 2024-04-25T01:20:50.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 124729 20.00 MB/sec execute 210 sec latency 131.049 ms 2024-04-25T01:20:51.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 125001 19.96 MB/sec execute 211 sec latency 121.961 ms 2024-04-25T01:20:52.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 125460 19.96 MB/sec execute 212 sec latency 84.365 ms 2024-04-25T01:20:53.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 125938 19.95 MB/sec execute 213 sec latency 125.873 ms 2024-04-25T01:20:54.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 126322 19.93 MB/sec execute 214 sec latency 336.883 ms 2024-04-25T01:20:55.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 126346 19.84 MB/sec execute 215 sec latency 1311.449 ms 2024-04-25T01:20:56.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 126558 19.81 MB/sec execute 216 sec latency 1271.022 ms 2024-04-25T01:20:57.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 126704 19.78 MB/sec execute 217 sec latency 273.403 ms 2024-04-25T01:20:58.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 127315 19.90 MB/sec execute 218 sec latency 95.828 ms 2024-04-25T01:20:59.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 127679 19.93 MB/sec execute 219 sec latency 134.978 ms 2024-04-25T01:21:00.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 127969 19.91 MB/sec execute 220 sec latency 119.401 ms 2024-04-25T01:21:01.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 128191 19.87 MB/sec execute 221 sec latency 113.585 ms 2024-04-25T01:21:02.172 INFO:tasks.workunit.client.1.smithi171.stdout: 10 128402 19.80 MB/sec execute 222 sec latency 125.322 ms 2024-04-25T01:21:03.173 INFO:tasks.workunit.client.1.smithi171.stdout: 10 128749 19.79 MB/sec execute 223 sec latency 112.393 ms 2024-04-25T01:21:04.173 INFO:tasks.workunit.client.1.smithi171.stdout: 10 129293 19.78 MB/sec execute 224 sec latency 119.699 ms 2024-04-25T01:21:05.173 INFO:tasks.workunit.client.1.smithi171.stdout: 10 129948 19.84 MB/sec execute 225 sec latency 51.682 ms 2024-04-25T01:21:06.173 INFO:tasks.workunit.client.1.smithi171.stdout: 10 130635 19.98 MB/sec execute 226 sec latency 46.010 ms 2024-04-25T01:21:07.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 131079 20.02 MB/sec execute 227 sec latency 104.029 ms 2024-04-25T01:21:08.174 INFO:tasks.workunit.client.1.smithi171.stdout: 10 131408 20.04 MB/sec execute 228 sec latency 97.058 ms 2024-04-25T01:21:09.174 INFO:tasks.workunit.client.1.smithi171.stdout: 10 131650 19.99 MB/sec execute 229 sec latency 109.175 ms 2024-04-25T01:21:10.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 131829 19.93 MB/sec execute 230 sec latency 196.977 ms 2024-04-25T01:21:11.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 132094 19.88 MB/sec execute 231 sec latency 104.256 ms 2024-04-25T01:21:12.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 132496 19.89 MB/sec execute 232 sec latency 108.746 ms 2024-04-25T01:21:13.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 133011 19.88 MB/sec execute 233 sec latency 123.359 ms 2024-04-25T01:21:14.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 133593 19.91 MB/sec execute 234 sec latency 80.385 ms 2024-04-25T01:21:15.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 133743 19.89 MB/sec execute 235 sec latency 820.755 ms 2024-04-25T01:21:16.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 133770 19.82 MB/sec execute 236 sec latency 1687.744 ms 2024-04-25T01:21:17.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 134024 19.81 MB/sec execute 237 sec latency 191.870 ms 2024-04-25T01:21:18.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 134323 19.81 MB/sec execute 238 sec latency 132.888 ms 2024-04-25T01:21:19.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 134563 19.79 MB/sec execute 239 sec latency 144.090 ms 2024-04-25T01:21:20.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 134916 19.82 MB/sec execute 240 sec latency 121.429 ms 2024-04-25T01:21:21.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 135167 19.79 MB/sec execute 241 sec latency 138.717 ms 2024-04-25T01:21:22.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 135375 19.73 MB/sec execute 242 sec latency 160.125 ms 2024-04-25T01:21:23.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 135648 19.69 MB/sec execute 243 sec latency 154.688 ms 2024-04-25T01:21:24.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 136096 19.70 MB/sec execute 244 sec latency 70.472 ms 2024-04-25T01:21:25.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 136593 19.68 MB/sec execute 245 sec latency 125.550 ms 2024-04-25T01:21:26.175 INFO:tasks.workunit.client.1.smithi171.stdout: 10 137230 19.75 MB/sec execute 246 sec latency 39.540 ms 2024-04-25T01:21:27.177 INFO:tasks.workunit.client.1.smithi171.stdout: 10 137906 19.86 MB/sec execute 247 sec latency 57.660 ms 2024-04-25T01:21:28.177 INFO:tasks.workunit.client.1.smithi171.stdout: 10 138245 19.89 MB/sec execute 248 sec latency 135.587 ms 2024-04-25T01:21:29.177 INFO:tasks.workunit.client.1.smithi171.stdout: 10 138539 19.88 MB/sec execute 249 sec latency 74.708 ms 2024-04-25T01:21:30.177 INFO:tasks.workunit.client.1.smithi171.stdout: 10 138760 19.85 MB/sec execute 250 sec latency 507.897 ms 2024-04-25T01:21:31.177 INFO:tasks.workunit.client.1.smithi171.stdout: 10 138780 19.77 MB/sec execute 251 sec latency 1173.353 ms 2024-04-25T01:21:32.177 INFO:tasks.workunit.client.1.smithi171.stdout: 10 138989 19.71 MB/sec execute 252 sec latency 441.236 ms 2024-04-25T01:21:33.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 139126 19.66 MB/sec execute 253 sec latency 139.045 ms 2024-04-25T01:21:34.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 139332 19.63 MB/sec execute 254 sec latency 154.020 ms 2024-04-25T01:21:35.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 139886 19.63 MB/sec execute 255 sec latency 87.690 ms 2024-04-25T01:21:36.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 140396 19.62 MB/sec execute 256 sec latency 151.611 ms 2024-04-25T01:21:37.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 141023 19.72 MB/sec execute 257 sec latency 52.834 ms 2024-04-25T01:21:38.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 141528 19.78 MB/sec execute 258 sec latency 138.100 ms 2024-04-25T01:21:39.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 141883 19.80 MB/sec execute 259 sec latency 96.447 ms 2024-04-25T01:21:40.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 142154 19.79 MB/sec execute 260 sec latency 96.005 ms 2024-04-25T01:21:41.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 142406 19.75 MB/sec execute 261 sec latency 71.314 ms 2024-04-25T01:21:42.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 142670 19.71 MB/sec execute 262 sec latency 168.940 ms 2024-04-25T01:21:43.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 143079 19.71 MB/sec execute 263 sec latency 75.932 ms 2024-04-25T01:21:44.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 143702 19.72 MB/sec execute 264 sec latency 67.233 ms 2024-04-25T01:21:45.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 144317 19.77 MB/sec execute 265 sec latency 76.421 ms 2024-04-25T01:21:46.179 INFO:tasks.workunit.client.1.smithi171.stdout: 10 144899 19.87 MB/sec execute 266 sec latency 114.750 ms 2024-04-25T01:21:47.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 145356 19.91 MB/sec execute 267 sec latency 89.747 ms 2024-04-25T01:21:48.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 145652 19.90 MB/sec execute 268 sec latency 134.292 ms 2024-04-25T01:21:49.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 145923 19.88 MB/sec execute 269 sec latency 122.437 ms 2024-04-25T01:21:50.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 146179 19.84 MB/sec execute 270 sec latency 110.055 ms 2024-04-25T01:21:51.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 146226 19.77 MB/sec execute 271 sec latency 839.909 ms 2024-04-25T01:21:52.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 146258 19.70 MB/sec execute 272 sec latency 1404.979 ms 2024-04-25T01:21:53.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 146377 19.64 MB/sec execute 273 sec latency 265.056 ms 2024-04-25T01:21:54.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 146570 19.60 MB/sec execute 274 sec latency 148.348 ms 2024-04-25T01:21:55.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 146865 19.56 MB/sec execute 275 sec latency 102.856 ms 2024-04-25T01:21:56.180 INFO:tasks.workunit.client.1.smithi171.stdout: 10 147413 19.59 MB/sec execute 276 sec latency 109.566 ms 2024-04-25T01:21:57.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 148096 19.67 MB/sec execute 277 sec latency 40.646 ms 2024-04-25T01:21:58.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 148753 19.76 MB/sec execute 278 sec latency 70.796 ms 2024-04-25T01:21:59.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 149098 19.79 MB/sec execute 279 sec latency 124.416 ms 2024-04-25T01:22:00.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 149410 19.77 MB/sec execute 280 sec latency 119.158 ms 2024-04-25T01:22:01.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 149653 19.73 MB/sec execute 281 sec latency 139.424 ms 2024-04-25T01:22:02.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 150009 19.72 MB/sec execute 282 sec latency 102.776 ms 2024-04-25T01:22:03.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 150501 19.69 MB/sec execute 283 sec latency 110.231 ms 2024-04-25T01:22:04.181 INFO:tasks.workunit.client.1.smithi171.stdout: 10 151116 19.75 MB/sec execute 284 sec latency 41.582 ms 2024-04-25T01:22:05.183 INFO:tasks.workunit.client.1.smithi171.stdout: 10 151689 19.80 MB/sec execute 285 sec latency 91.096 ms 2024-04-25T01:22:06.183 INFO:tasks.workunit.client.1.smithi171.stdout: 10 151973 19.82 MB/sec execute 286 sec latency 629.586 ms 2024-04-25T01:22:07.183 INFO:tasks.workunit.client.1.smithi171.stdout: 10 151997 19.76 MB/sec execute 287 sec latency 1176.964 ms 2024-04-25T01:22:08.183 INFO:tasks.workunit.client.1.smithi171.stdout: 10 152196 19.72 MB/sec execute 288 sec latency 214.526 ms 2024-04-25T01:22:09.183 INFO:tasks.workunit.client.1.smithi171.stdout: 10 152503 19.73 MB/sec execute 289 sec latency 114.559 ms 2024-04-25T01:22:10.183 INFO:tasks.workunit.client.1.smithi171.stdout: 10 152809 19.72 MB/sec execute 290 sec latency 90.094 ms 2024-04-25T01:22:11.184 INFO:tasks.workunit.client.1.smithi171.stdout: 10 153041 19.68 MB/sec execute 291 sec latency 131.378 ms 2024-04-25T01:22:12.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 153319 19.65 MB/sec execute 292 sec latency 166.336 ms 2024-04-25T01:22:13.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 153671 19.64 MB/sec execute 293 sec latency 126.396 ms 2024-04-25T01:22:14.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 154252 19.66 MB/sec execute 294 sec latency 92.472 ms 2024-04-25T01:22:15.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 154918 19.70 MB/sec execute 295 sec latency 75.458 ms 2024-04-25T01:22:16.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 155496 19.80 MB/sec execute 296 sec latency 71.765 ms 2024-04-25T01:22:17.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 155922 19.79 MB/sec execute 297 sec latency 84.616 ms 2024-04-25T01:22:18.185 INFO:tasks.workunit.client.1.smithi171.stdout: 10 156285 19.82 MB/sec execute 298 sec latency 62.776 ms 2024-04-25T01:22:19.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 156484 19.77 MB/sec execute 299 sec latency 178.223 ms 2024-04-25T01:22:20.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 156872 19.78 MB/sec execute 300 sec latency 96.093 ms 2024-04-25T01:22:21.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 157202 19.75 MB/sec execute 301 sec latency 109.874 ms 2024-04-25T01:22:22.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 157741 19.76 MB/sec execute 302 sec latency 151.053 ms 2024-04-25T01:22:23.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 158437 19.81 MB/sec execute 303 sec latency 76.528 ms 2024-04-25T01:22:24.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 159025 19.91 MB/sec execute 304 sec latency 76.766 ms 2024-04-25T01:22:25.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 159521 19.92 MB/sec execute 305 sec latency 121.625 ms 2024-04-25T01:22:25.218 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:22:25.221 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:22:26.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 159853 19.92 MB/sec execute 306 sec latency 194.934 ms 2024-04-25T01:22:27.189 INFO:tasks.workunit.client.1.smithi171.stdout: 10 159867 19.86 MB/sec execute 307 sec latency 1165.046 ms 2024-04-25T01:22:28.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 159900 19.80 MB/sec execute 308 sec latency 1828.976 ms 2024-04-25T01:22:29.195 INFO:tasks.workunit.client.1.smithi171.stdout: 10 160022 19.75 MB/sec execute 309 sec latency 148.948 ms 2024-04-25T01:22:30.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 160212 19.71 MB/sec execute 310 sec latency 98.999 ms 2024-04-25T01:22:31.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 160521 19.71 MB/sec execute 311 sec latency 137.263 ms 2024-04-25T01:22:32.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 160942 19.68 MB/sec execute 312 sec latency 174.676 ms 2024-04-25T01:22:33.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 161483 19.71 MB/sec execute 313 sec latency 85.383 ms 2024-04-25T01:22:34.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 162061 19.73 MB/sec execute 314 sec latency 132.313 ms 2024-04-25T01:22:35.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 162639 19.83 MB/sec execute 315 sec latency 50.707 ms 2024-04-25T01:22:36.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 163140 19.85 MB/sec execute 316 sec latency 105.048 ms 2024-04-25T01:22:37.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 163398 19.84 MB/sec execute 317 sec latency 139.328 ms 2024-04-25T01:22:38.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 163730 19.82 MB/sec execute 318 sec latency 88.026 ms 2024-04-25T01:22:39.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 164126 19.82 MB/sec execute 319 sec latency 73.253 ms 2024-04-25T01:22:40.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 164474 19.79 MB/sec execute 320 sec latency 145.036 ms 2024-04-25T01:22:41.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 165004 19.82 MB/sec execute 321 sec latency 90.315 ms 2024-04-25T01:22:42.191 INFO:tasks.workunit.client.1.smithi171.stdout: 10 165675 19.86 MB/sec execute 322 sec latency 94.468 ms 2024-04-25T01:22:43.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 166268 19.94 MB/sec execute 323 sec latency 50.248 ms 2024-04-25T01:22:44.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 166753 19.96 MB/sec execute 324 sec latency 74.101 ms 2024-04-25T01:22:45.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 167039 19.95 MB/sec execute 325 sec latency 99.276 ms 2024-04-25T01:22:46.192 INFO:tasks.workunit.client.1.smithi171.stdout: 10 167324 19.94 MB/sec execute 326 sec latency 151.776 ms 2024-04-25T01:22:47.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 167438 19.88 MB/sec execute 327 sec latency 669.668 ms 2024-04-25T01:22:48.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 167461 19.82 MB/sec execute 328 sec latency 1582.047 ms 2024-04-25T01:22:49.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 167602 19.78 MB/sec execute 329 sec latency 619.306 ms 2024-04-25T01:22:50.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 167791 19.75 MB/sec execute 330 sec latency 173.115 ms 2024-04-25T01:22:51.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 168048 19.72 MB/sec execute 331 sec latency 223.049 ms 2024-04-25T01:22:52.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 168606 19.74 MB/sec execute 332 sec latency 54.888 ms 2024-04-25T01:22:53.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 169102 19.76 MB/sec execute 333 sec latency 189.008 ms 2024-04-25T01:22:54.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 169601 19.82 MB/sec execute 334 sec latency 91.011 ms 2024-04-25T01:22:55.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 170100 19.83 MB/sec execute 335 sec latency 147.007 ms 2024-04-25T01:22:55.506 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:22:55.509 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:22:55.539 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:22:55.558+0000 7f224640f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41802) UID: 0 2024-04-25T01:22:55.539 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:22:55.558+0000 7ff199406640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41802) UID: 0 2024-04-25T01:22:56.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 170462 19.85 MB/sec execute 336 sec latency 65.143 ms 2024-04-25T01:22:57.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 170675 19.81 MB/sec execute 337 sec latency 201.093 ms 2024-04-25T01:22:58.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 170971 19.81 MB/sec execute 338 sec latency 127.303 ms 2024-04-25T01:22:59.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 171278 19.79 MB/sec execute 339 sec latency 145.590 ms 2024-04-25T01:23:00.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 171688 19.76 MB/sec execute 340 sec latency 114.426 ms 2024-04-25T01:23:01.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 172066 19.78 MB/sec execute 341 sec latency 185.416 ms 2024-04-25T01:23:02.193 INFO:tasks.workunit.client.1.smithi171.stdout: 10 172651 19.80 MB/sec execute 342 sec latency 113.669 ms 2024-04-25T01:23:03.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 173121 19.85 MB/sec execute 343 sec latency 110.999 ms 2024-04-25T01:23:04.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 173591 19.86 MB/sec execute 344 sec latency 125.651 ms 2024-04-25T01:23:05.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 173896 19.86 MB/sec execute 345 sec latency 137.016 ms 2024-04-25T01:23:06.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 174195 19.85 MB/sec execute 346 sec latency 119.450 ms 2024-04-25T01:23:07.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 174386 19.82 MB/sec execute 347 sec latency 283.805 ms 2024-04-25T01:23:08.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 174409 19.77 MB/sec execute 348 sec latency 778.171 ms 2024-04-25T01:23:09.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 174449 19.72 MB/sec execute 349 sec latency 1646.190 ms 2024-04-25T01:23:10.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 174625 19.67 MB/sec execute 350 sec latency 1776.469 ms 2024-04-25T01:23:11.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 174816 19.65 MB/sec execute 351 sec latency 140.785 ms 2024-04-25T01:23:12.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 175011 19.61 MB/sec execute 352 sec latency 246.778 ms 2024-04-25T01:23:13.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 175586 19.64 MB/sec execute 353 sec latency 39.055 ms 2024-04-25T01:23:14.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 176180 19.66 MB/sec execute 354 sec latency 96.795 ms 2024-04-25T01:23:15.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 176635 19.70 MB/sec execute 355 sec latency 113.694 ms 2024-04-25T01:23:16.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 177185 19.73 MB/sec execute 356 sec latency 91.052 ms 2024-04-25T01:23:17.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 177450 19.73 MB/sec execute 357 sec latency 143.818 ms 2024-04-25T01:23:18.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 177777 19.72 MB/sec execute 358 sec latency 85.494 ms 2024-04-25T01:23:19.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 178109 19.71 MB/sec execute 359 sec latency 104.307 ms 2024-04-25T01:23:20.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 178473 19.70 MB/sec execute 360 sec latency 72.491 ms 2024-04-25T01:23:21.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 179006 19.71 MB/sec execute 361 sec latency 65.930 ms 2024-04-25T01:23:22.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 179482 19.74 MB/sec execute 362 sec latency 76.653 ms 2024-04-25T01:23:23.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 180112 19.78 MB/sec execute 363 sec latency 37.620 ms 2024-04-25T01:23:24.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 180581 19.81 MB/sec execute 364 sec latency 96.284 ms 2024-04-25T01:23:25.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 180955 19.82 MB/sec execute 365 sec latency 102.778 ms 2024-04-25T01:23:26.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 181261 19.81 MB/sec execute 366 sec latency 86.233 ms 2024-04-25T01:23:27.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 181604 19.80 MB/sec execute 367 sec latency 96.764 ms 2024-04-25T01:23:28.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 181914 19.78 MB/sec execute 368 sec latency 104.172 ms 2024-04-25T01:23:29.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 182390 19.78 MB/sec execute 369 sec latency 123.998 ms 2024-04-25T01:23:30.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 182889 19.81 MB/sec execute 370 sec latency 92.726 ms 2024-04-25T01:23:31.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 183541 19.85 MB/sec execute 371 sec latency 81.795 ms 2024-04-25T01:23:32.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184047 19.88 MB/sec execute 372 sec latency 73.349 ms 2024-04-25T01:23:33.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184242 19.85 MB/sec execute 373 sec latency 607.232 ms 2024-04-25T01:23:34.194 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184247 19.80 MB/sec execute 374 sec latency 1295.298 ms 2024-04-25T01:23:35.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184262 19.75 MB/sec execute 375 sec latency 2172.448 ms 2024-04-25T01:23:36.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184438 19.72 MB/sec execute 376 sec latency 507.423 ms 2024-04-25T01:23:37.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184619 19.71 MB/sec execute 377 sec latency 203.128 ms 2024-04-25T01:23:38.200 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184800 19.69 MB/sec execute 378 sec latency 93.822 ms 2024-04-25T01:23:39.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 184964 19.66 MB/sec execute 379 sec latency 120.993 ms 2024-04-25T01:23:40.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 185294 19.64 MB/sec execute 380 sec latency 80.917 ms 2024-04-25T01:23:41.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 185589 19.63 MB/sec execute 381 sec latency 125.118 ms 2024-04-25T01:23:42.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 186090 19.63 MB/sec execute 382 sec latency 85.361 ms 2024-04-25T01:23:43.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 186551 19.65 MB/sec execute 383 sec latency 114.735 ms 2024-04-25T01:23:44.198 INFO:tasks.workunit.client.1.smithi171.stdout: 10 187179 19.68 MB/sec execute 384 sec latency 51.226 ms 2024-04-25T01:23:45.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 187670 19.72 MB/sec execute 385 sec latency 97.389 ms 2024-04-25T01:23:46.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 188072 19.74 MB/sec execute 386 sec latency 96.560 ms 2024-04-25T01:23:47.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 188352 19.72 MB/sec execute 387 sec latency 93.488 ms 2024-04-25T01:23:48.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 188602 19.70 MB/sec execute 388 sec latency 262.847 ms 2024-04-25T01:23:49.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 188629 19.65 MB/sec execute 389 sec latency 1097.564 ms 2024-04-25T01:23:50.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 188871 19.63 MB/sec execute 390 sec latency 387.274 ms 2024-04-25T01:23:51.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 189054 19.60 MB/sec execute 391 sec latency 156.346 ms 2024-04-25T01:23:52.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 189354 19.59 MB/sec execute 392 sec latency 141.483 ms 2024-04-25T01:23:53.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 189848 19.59 MB/sec execute 393 sec latency 94.609 ms 2024-04-25T01:23:54.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 190416 19.62 MB/sec execute 394 sec latency 88.518 ms 2024-04-25T01:23:55.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 190892 19.65 MB/sec execute 395 sec latency 66.266 ms 2024-04-25T01:23:56.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 191351 19.67 MB/sec execute 396 sec latency 83.684 ms 2024-04-25T01:23:57.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 191705 19.68 MB/sec execute 397 sec latency 77.752 ms 2024-04-25T01:23:58.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 192005 19.68 MB/sec execute 398 sec latency 106.919 ms 2024-04-25T01:23:59.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 192334 19.65 MB/sec execute 399 sec latency 98.485 ms 2024-04-25T01:24:00.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 192644 19.65 MB/sec execute 400 sec latency 136.088 ms 2024-04-25T01:24:01.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 193195 19.65 MB/sec execute 401 sec latency 73.667 ms 2024-04-25T01:24:02.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 193640 19.66 MB/sec execute 402 sec latency 129.672 ms 2024-04-25T01:24:03.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 194230 19.70 MB/sec execute 403 sec latency 48.349 ms 2024-04-25T01:24:04.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 194735 19.74 MB/sec execute 404 sec latency 100.751 ms 2024-04-25T01:24:05.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 195111 19.75 MB/sec execute 405 sec latency 98.312 ms 2024-04-25T01:24:06.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 195363 19.73 MB/sec execute 406 sec latency 148.086 ms 2024-04-25T01:24:07.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 195652 19.72 MB/sec execute 407 sec latency 129.101 ms 2024-04-25T01:24:08.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 195984 19.70 MB/sec execute 408 sec latency 153.032 ms 2024-04-25T01:24:09.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 196083 19.67 MB/sec execute 409 sec latency 770.286 ms 2024-04-25T01:24:10.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 196096 19.62 MB/sec execute 410 sec latency 1459.569 ms 2024-04-25T01:24:11.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 196238 19.59 MB/sec execute 411 sec latency 1499.069 ms 2024-04-25T01:24:12.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 196469 19.55 MB/sec execute 412 sec latency 178.547 ms 2024-04-25T01:24:13.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 196755 19.55 MB/sec execute 413 sec latency 113.293 ms 2024-04-25T01:24:14.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 197114 19.55 MB/sec execute 414 sec latency 110.773 ms 2024-04-25T01:24:15.196 INFO:tasks.workunit.client.1.smithi171.stdout: 10 197681 19.57 MB/sec execute 415 sec latency 101.107 ms 2024-04-25T01:24:16.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 198142 19.61 MB/sec execute 416 sec latency 93.181 ms 2024-04-25T01:24:17.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 198520 19.61 MB/sec execute 417 sec latency 93.355 ms 2024-04-25T01:24:18.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 198874 19.61 MB/sec execute 418 sec latency 83.137 ms 2024-04-25T01:24:19.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 199069 19.59 MB/sec execute 419 sec latency 153.141 ms 2024-04-25T01:24:20.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 199397 19.58 MB/sec execute 420 sec latency 91.140 ms 2024-04-25T01:24:21.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 199781 19.57 MB/sec execute 421 sec latency 123.482 ms 2024-04-25T01:24:22.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 200289 19.58 MB/sec execute 422 sec latency 67.668 ms 2024-04-25T01:24:23.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 200837 19.61 MB/sec execute 423 sec latency 95.776 ms 2024-04-25T01:24:24.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 201359 19.64 MB/sec execute 424 sec latency 118.463 ms 2024-04-25T01:24:25.197 INFO:tasks.workunit.client.1.smithi171.stdout: 10 201839 19.66 MB/sec execute 425 sec latency 127.520 ms 2024-04-25T01:24:26.198 INFO:tasks.workunit.client.1.smithi171.stdout: 10 202287 19.68 MB/sec execute 426 sec latency 73.042 ms 2024-04-25T01:24:27.200 INFO:tasks.workunit.client.1.smithi171.stdout: 10 202550 19.66 MB/sec execute 427 sec latency 92.915 ms 2024-04-25T01:24:28.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 202872 19.66 MB/sec execute 428 sec latency 104.004 ms 2024-04-25T01:24:29.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 203258 19.65 MB/sec execute 429 sec latency 94.725 ms 2024-04-25T01:24:30.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 203777 19.65 MB/sec execute 430 sec latency 98.263 ms 2024-04-25T01:24:31.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 204340 19.69 MB/sec execute 431 sec latency 72.595 ms 2024-04-25T01:24:32.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 204863 19.71 MB/sec execute 432 sec latency 86.467 ms 2024-04-25T01:24:33.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 205408 19.75 MB/sec execute 433 sec latency 66.784 ms 2024-04-25T01:24:34.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 205766 19.75 MB/sec execute 434 sec latency 204.960 ms 2024-04-25T01:24:35.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 205768 19.70 MB/sec execute 435 sec latency 1203.097 ms 2024-04-25T01:24:36.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 205779 19.66 MB/sec execute 436 sec latency 1466.381 ms 2024-04-25T01:24:37.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 205837 19.62 MB/sec execute 437 sec latency 1620.052 ms 2024-04-25T01:24:38.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 205980 19.59 MB/sec execute 438 sec latency 175.522 ms 2024-04-25T01:24:39.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 206144 19.56 MB/sec execute 439 sec latency 113.853 ms 2024-04-25T01:24:40.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 206341 19.54 MB/sec execute 440 sec latency 111.111 ms 2024-04-25T01:24:41.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 206543 19.52 MB/sec execute 441 sec latency 166.297 ms 2024-04-25T01:24:42.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 206984 19.52 MB/sec execute 442 sec latency 83.053 ms 2024-04-25T01:24:43.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 207398 19.52 MB/sec execute 443 sec latency 141.881 ms 2024-04-25T01:24:44.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 207906 19.54 MB/sec execute 444 sec latency 97.233 ms 2024-04-25T01:24:45.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 208372 19.56 MB/sec execute 445 sec latency 115.120 ms 2024-04-25T01:24:46.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 208735 19.57 MB/sec execute 446 sec latency 177.920 ms 2024-04-25T01:24:47.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 209080 19.57 MB/sec execute 447 sec latency 144.099 ms 2024-04-25T01:24:48.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 209451 19.58 MB/sec execute 448 sec latency 92.872 ms 2024-04-25T01:24:49.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 209731 19.56 MB/sec execute 449 sec latency 95.843 ms 2024-04-25T01:24:50.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 209819 19.53 MB/sec execute 450 sec latency 805.144 ms 2024-04-25T01:24:51.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 209885 19.49 MB/sec execute 451 sec latency 1172.127 ms 2024-04-25T01:24:52.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 210082 19.47 MB/sec execute 452 sec latency 171.188 ms 2024-04-25T01:24:53.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 210513 19.46 MB/sec execute 453 sec latency 119.081 ms 2024-04-25T01:24:54.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 210959 19.47 MB/sec execute 454 sec latency 171.706 ms 2024-04-25T01:24:55.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 211512 19.49 MB/sec execute 455 sec latency 93.207 ms 2024-04-25T01:24:56.201 INFO:tasks.workunit.client.1.smithi171.stdout: 10 212012 19.52 MB/sec execute 456 sec latency 97.995 ms 2024-04-25T01:24:57.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 212434 19.54 MB/sec execute 457 sec latency 133.226 ms 2024-04-25T01:24:58.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 212761 19.54 MB/sec execute 458 sec latency 164.388 ms 2024-04-25T01:24:59.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 213060 19.53 MB/sec execute 459 sec latency 142.229 ms 2024-04-25T01:25:00.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 213143 19.49 MB/sec execute 460 sec latency 608.186 ms 2024-04-25T01:25:01.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 213294 19.47 MB/sec execute 461 sec latency 1094.625 ms 2024-04-25T01:25:02.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 213498 19.45 MB/sec execute 462 sec latency 158.906 ms 2024-04-25T01:25:03.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 213839 19.44 MB/sec execute 463 sec latency 107.956 ms 2024-04-25T01:25:04.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 214316 19.44 MB/sec execute 464 sec latency 122.482 ms 2024-04-25T01:25:05.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 214862 19.46 MB/sec execute 465 sec latency 105.773 ms 2024-04-25T01:25:06.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 215392 19.48 MB/sec execute 466 sec latency 116.415 ms 2024-04-25T01:25:07.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 215764 19.49 MB/sec execute 467 sec latency 158.166 ms 2024-04-25T01:25:08.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 216201 19.51 MB/sec execute 468 sec latency 102.216 ms 2024-04-25T01:25:09.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 216544 19.51 MB/sec execute 469 sec latency 99.996 ms 2024-04-25T01:25:10.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 216796 19.49 MB/sec execute 470 sec latency 138.323 ms 2024-04-25T01:25:11.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 217136 19.48 MB/sec execute 471 sec latency 103.664 ms 2024-04-25T01:25:12.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 217494 19.48 MB/sec execute 472 sec latency 136.956 ms 2024-04-25T01:25:13.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 218003 19.48 MB/sec execute 473 sec latency 111.177 ms 2024-04-25T01:25:14.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 218545 19.50 MB/sec execute 474 sec latency 92.430 ms 2024-04-25T01:25:15.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 219106 19.54 MB/sec execute 475 sec latency 67.627 ms 2024-04-25T01:25:16.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 219533 19.55 MB/sec execute 476 sec latency 116.369 ms 2024-04-25T01:25:17.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 219943 19.57 MB/sec execute 477 sec latency 81.862 ms 2024-04-25T01:25:18.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 220171 19.55 MB/sec execute 478 sec latency 141.112 ms 2024-04-25T01:25:19.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 220502 19.54 MB/sec execute 479 sec latency 131.305 ms 2024-04-25T01:25:20.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 220857 19.54 MB/sec execute 480 sec latency 120.375 ms 2024-04-25T01:25:21.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 221320 19.53 MB/sec execute 481 sec latency 98.732 ms 2024-04-25T01:25:22.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 221823 19.54 MB/sec execute 482 sec latency 65.394 ms 2024-04-25T01:25:23.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 222359 19.57 MB/sec execute 483 sec latency 106.602 ms 2024-04-25T01:25:24.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 222811 19.59 MB/sec execute 484 sec latency 103.306 ms 2024-04-25T01:25:25.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 223269 19.62 MB/sec execute 485 sec latency 130.274 ms 2024-04-25T01:25:26.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 223603 19.61 MB/sec execute 486 sec latency 104.145 ms 2024-04-25T01:25:27.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 223798 19.59 MB/sec execute 487 sec latency 304.444 ms 2024-04-25T01:25:28.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 224082 19.58 MB/sec execute 488 sec latency 101.136 ms 2024-04-25T01:25:29.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 224420 19.58 MB/sec execute 489 sec latency 115.696 ms 2024-04-25T01:25:30.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 224912 19.56 MB/sec execute 490 sec latency 114.680 ms 2024-04-25T01:25:31.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 225363 19.58 MB/sec execute 491 sec latency 99.964 ms 2024-04-25T01:25:32.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 225963 19.60 MB/sec execute 492 sec latency 70.955 ms 2024-04-25T01:25:33.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 226490 19.64 MB/sec execute 493 sec latency 88.292 ms 2024-04-25T01:25:34.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 226945 19.66 MB/sec execute 494 sec latency 60.133 ms 2024-04-25T01:25:35.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227251 19.65 MB/sec execute 495 sec latency 94.326 ms 2024-04-25T01:25:36.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227344 19.62 MB/sec execute 496 sec latency 710.913 ms 2024-04-25T01:25:37.205 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227344 19.58 MB/sec execute 497 sec latency 1679.027 ms 2024-04-25T01:25:38.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227350 19.54 MB/sec execute 498 sec latency 2506.538 ms 2024-04-25T01:25:39.203 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227367 19.50 MB/sec execute 499 sec latency 2787.034 ms 2024-04-25T01:25:40.202 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227472 19.48 MB/sec execute 500 sec latency 1551.728 ms 2024-04-25T01:25:41.203 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227643 19.45 MB/sec execute 501 sec latency 210.655 ms 2024-04-25T01:25:42.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 227777 19.42 MB/sec execute 502 sec latency 215.515 ms 2024-04-25T01:25:43.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 228000 19.42 MB/sec execute 503 sec latency 173.464 ms 2024-04-25T01:25:44.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 228224 19.39 MB/sec execute 504 sec latency 131.315 ms 2024-04-25T01:25:45.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 228595 19.38 MB/sec execute 505 sec latency 95.360 ms 2024-04-25T01:25:46.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 228946 19.37 MB/sec execute 506 sec latency 125.999 ms 2024-04-25T01:25:47.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 229471 19.39 MB/sec execute 507 sec latency 162.015 ms 2024-04-25T01:25:48.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 230043 19.44 MB/sec execute 508 sec latency 62.642 ms 2024-04-25T01:25:49.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 230450 19.45 MB/sec execute 509 sec latency 104.794 ms 2024-04-25T01:25:50.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 230737 19.45 MB/sec execute 510 sec latency 110.353 ms 2024-04-25T01:25:51.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 230919 19.42 MB/sec execute 511 sec latency 292.902 ms 2024-04-25T01:25:52.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 230955 19.39 MB/sec execute 512 sec latency 1166.507 ms 2024-04-25T01:25:53.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 231110 19.36 MB/sec execute 513 sec latency 1094.420 ms 2024-04-25T01:25:54.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 231302 19.34 MB/sec execute 514 sec latency 169.811 ms 2024-04-25T01:25:55.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 231703 19.35 MB/sec execute 515 sec latency 106.947 ms 2024-04-25T01:25:56.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 232369 19.34 MB/sec execute 516 sec latency 72.093 ms 2024-04-25T01:25:57.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 232841 19.37 MB/sec execute 517 sec latency 96.032 ms 2024-04-25T01:25:58.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 233469 19.41 MB/sec execute 518 sec latency 82.399 ms 2024-04-25T01:25:59.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 233930 19.44 MB/sec execute 519 sec latency 113.912 ms 2024-04-25T01:26:00.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 234269 19.44 MB/sec execute 520 sec latency 100.520 ms 2024-04-25T01:26:01.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 234534 19.42 MB/sec execute 521 sec latency 92.639 ms 2024-04-25T01:26:02.210 INFO:tasks.workunit.client.1.smithi171.stdout: 10 234650 19.39 MB/sec execute 522 sec latency 666.351 ms 2024-04-25T01:26:03.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 234720 19.36 MB/sec execute 523 sec latency 985.263 ms 2024-04-25T01:26:04.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 234943 19.34 MB/sec execute 524 sec latency 228.611 ms 2024-04-25T01:26:05.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 235367 19.35 MB/sec execute 525 sec latency 89.038 ms 2024-04-25T01:26:06.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 235992 19.36 MB/sec execute 526 sec latency 103.145 ms 2024-04-25T01:26:07.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 236448 19.37 MB/sec execute 527 sec latency 123.330 ms 2024-04-25T01:26:08.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 237011 19.41 MB/sec execute 528 sec latency 113.970 ms 2024-04-25T01:26:09.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 237498 19.43 MB/sec execute 529 sec latency 99.830 ms 2024-04-25T01:26:10.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 237827 19.43 MB/sec execute 530 sec latency 90.097 ms 2024-04-25T01:26:11.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 238066 19.41 MB/sec execute 531 sec latency 169.872 ms 2024-04-25T01:26:12.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 238366 19.40 MB/sec execute 532 sec latency 161.741 ms 2024-04-25T01:26:13.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 238756 19.41 MB/sec execute 533 sec latency 118.387 ms 2024-04-25T01:26:14.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 239334 19.40 MB/sec execute 534 sec latency 119.461 ms 2024-04-25T01:26:15.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 239868 19.43 MB/sec execute 535 sec latency 107.428 ms 2024-04-25T01:26:16.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 240544 19.47 MB/sec execute 536 sec latency 67.506 ms 2024-04-25T01:26:17.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 240995 19.50 MB/sec execute 537 sec latency 131.295 ms 2024-04-25T01:26:18.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 241371 19.50 MB/sec execute 538 sec latency 70.949 ms 2024-04-25T01:26:19.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 241642 19.49 MB/sec execute 539 sec latency 126.161 ms 2024-04-25T01:26:20.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 241942 19.47 MB/sec execute 540 sec latency 138.634 ms 2024-04-25T01:26:21.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 242384 19.49 MB/sec execute 541 sec latency 81.996 ms 2024-04-25T01:26:22.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 243022 19.48 MB/sec execute 542 sec latency 84.901 ms 2024-04-25T01:26:22.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:26:22.227 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:26:22.259 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:26:22.279+0000 7f224640f640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41829) UID: 0 2024-04-25T01:26:22.259 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:26:22.279+0000 7ff199406640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 41829) UID: 0 2024-04-25T01:26:22.393 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:26:22.414+0000 7fce923c8640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46419) UID: 0 2024-04-25T01:26:22.394 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:26:22.414+0000 7f1f46d2c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46419) UID: 0 2024-04-25T01:26:22.394 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:26:22.414+0000 7fd7e31ad640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46419) UID: 0 2024-04-25T01:26:22.394 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:26:22.414+0000 7f309042f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46419) UID: 0 2024-04-25T01:26:23.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 243550 19.51 MB/sec execute 543 sec latency 97.398 ms 2024-04-25T01:26:24.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 244221 19.56 MB/sec execute 544 sec latency 52.189 ms 2024-04-25T01:26:25.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 244625 19.57 MB/sec execute 545 sec latency 118.406 ms 2024-04-25T01:26:26.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 244939 19.57 MB/sec execute 546 sec latency 81.863 ms 2024-04-25T01:26:27.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 245183 19.55 MB/sec execute 547 sec latency 181.321 ms 2024-04-25T01:26:28.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 245356 19.53 MB/sec execute 548 sec latency 224.270 ms 2024-04-25T01:26:29.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 245674 19.52 MB/sec execute 549 sec latency 131.382 ms 2024-04-25T01:26:30.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 246091 19.52 MB/sec execute 550 sec latency 107.849 ms 2024-04-25T01:26:31.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 246685 19.52 MB/sec execute 551 sec latency 92.860 ms 2024-04-25T01:26:32.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 247310 19.56 MB/sec execute 552 sec latency 71.322 ms 2024-04-25T01:26:33.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 247899 19.60 MB/sec execute 553 sec latency 115.795 ms 2024-04-25T01:26:34.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 248308 19.61 MB/sec execute 554 sec latency 61.987 ms 2024-04-25T01:26:35.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 248623 19.61 MB/sec execute 555 sec latency 144.733 ms 2024-04-25T01:26:36.211 INFO:tasks.workunit.client.1.smithi171.stdout: 10 248841 19.59 MB/sec execute 556 sec latency 177.769 ms 2024-04-25T01:26:37.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249098 19.57 MB/sec execute 557 sec latency 112.154 ms 2024-04-25T01:26:38.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249141 19.54 MB/sec execute 558 sec latency 818.346 ms 2024-04-25T01:26:39.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249150 19.51 MB/sec execute 559 sec latency 1505.191 ms 2024-04-25T01:26:40.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249150 19.47 MB/sec execute 560 sec latency 1287.996 ms 2024-04-25T01:26:41.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249163 19.44 MB/sec execute 561 sec latency 1790.474 ms 2024-04-25T01:26:42.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249304 19.42 MB/sec execute 562 sec latency 842.857 ms 2024-04-25T01:26:43.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249501 19.40 MB/sec execute 563 sec latency 262.931 ms 2024-04-25T01:26:44.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 249788 19.38 MB/sec execute 564 sec latency 202.139 ms 2024-04-25T01:26:45.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 250071 19.36 MB/sec execute 565 sec latency 172.631 ms 2024-04-25T01:26:46.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 250275 19.34 MB/sec execute 566 sec latency 161.124 ms 2024-04-25T01:26:47.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 250475 19.34 MB/sec execute 567 sec latency 155.262 ms 2024-04-25T01:26:48.212 INFO:tasks.workunit.client.1.smithi171.stdout: 10 250919 19.35 MB/sec execute 568 sec latency 292.212 ms 2024-04-25T01:26:49.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 251484 19.38 MB/sec execute 569 sec latency 96.034 ms 2024-04-25T01:26:50.215 INFO:tasks.workunit.client.1.smithi171.stdout: 10 251820 19.39 MB/sec execute 570 sec latency 127.138 ms 2024-04-25T01:26:51.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 252168 19.39 MB/sec execute 571 sec latency 72.867 ms 2024-04-25T01:26:52.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 252373 19.38 MB/sec execute 572 sec latency 146.126 ms 2024-04-25T01:26:53.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 252635 19.36 MB/sec execute 573 sec latency 159.324 ms 2024-04-25T01:26:54.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 253090 19.37 MB/sec execute 574 sec latency 93.464 ms 2024-04-25T01:26:55.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 253580 19.35 MB/sec execute 575 sec latency 124.289 ms 2024-04-25T01:26:56.213 INFO:tasks.workunit.client.1.smithi171.stdout: 10 254161 19.39 MB/sec execute 576 sec latency 90.961 ms 2024-04-25T01:26:57.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 254782 19.41 MB/sec execute 577 sec latency 92.498 ms 2024-04-25T01:26:58.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 255071 19.41 MB/sec execute 578 sec latency 305.758 ms 2024-04-25T01:26:59.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 255083 19.38 MB/sec execute 579 sec latency 1305.796 ms 2024-04-25T01:27:00.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 255206 19.37 MB/sec execute 580 sec latency 1841.373 ms 2024-04-25T01:27:01.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 255347 19.35 MB/sec execute 581 sec latency 151.699 ms 2024-04-25T01:27:02.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 255589 19.34 MB/sec execute 582 sec latency 94.843 ms 2024-04-25T01:27:03.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 255833 19.33 MB/sec execute 583 sec latency 106.910 ms 2024-04-25T01:27:04.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 256041 19.31 MB/sec execute 584 sec latency 126.951 ms 2024-04-25T01:27:05.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 256292 19.30 MB/sec execute 585 sec latency 202.576 ms 2024-04-25T01:27:06.221 INFO:tasks.workunit.client.1.smithi171.stdout: 10 256738 19.30 MB/sec execute 586 sec latency 67.290 ms 2024-04-25T01:27:07.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 257010 19.27 MB/sec execute 587 sec latency 472.412 ms 2024-04-25T01:27:08.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 257622 19.30 MB/sec execute 588 sec latency 46.254 ms 2024-04-25T01:27:09.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 258278 19.33 MB/sec execute 589 sec latency 62.788 ms 2024-04-25T01:27:10.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 258702 19.36 MB/sec execute 590 sec latency 166.049 ms 2024-04-25T01:27:11.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 259022 19.36 MB/sec execute 591 sec latency 91.459 ms 2024-04-25T01:27:12.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 259355 19.36 MB/sec execute 592 sec latency 86.815 ms 2024-04-25T01:27:13.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 259527 19.34 MB/sec execute 593 sec latency 132.793 ms 2024-04-25T01:27:14.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 259537 19.31 MB/sec execute 594 sec latency 976.636 ms 2024-04-25T01:27:15.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 259675 19.28 MB/sec execute 595 sec latency 1431.582 ms 2024-04-25T01:27:16.219 INFO:tasks.workunit.client.1.smithi171.stdout: 10 259853 19.26 MB/sec execute 596 sec latency 128.748 ms 2024-04-25T01:27:17.218 INFO:tasks.workunit.client.1.smithi171.stdout: 10 260125 19.25 MB/sec execute 597 sec latency 125.769 ms 2024-04-25T01:27:18.220 INFO:tasks.workunit.client.1.smithi171.stdout: 10 260579 19.25 MB/sec execute 598 sec latency 137.748 ms 2024-04-25T01:27:19.228 INFO:tasks.workunit.client.1.smithi171.stdout: 10 261206 19.26 MB/sec execute 599 sec latency 85.909 ms 2024-04-25T01:27:20.228 INFO:tasks.workunit.client.1.smithi171.stdout: 10 cleanup 600 sec 2024-04-25T01:27:21.232 INFO:tasks.workunit.client.1.smithi171.stdout: 10 cleanup 601 sec 2024-04-25T01:27:21.862 INFO:tasks.workunit.client.1.smithi171.stdout: 0 cleanup 602 sec 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Operation Count AvgLat MaxLat 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: ---------------------------------------- 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: NTCreateX 371761 6.412 2506.531 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Close 273261 0.003 0.099 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Rename 15764 25.523 2286.305 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Unlink 74930 9.120 659.244 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Qpathinfo 337452 4.847 2501.932 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Qfileinfo 58900 0.001 0.023 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Qfsinfo 61651 0.256 34.544 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Sfileinfo 30389 0.991 202.341 2024-04-25T01:27:21.863 INFO:tasks.workunit.client.1.smithi171.stdout: Find 130264 0.563 469.894 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout: WriteX 183823 0.333 2167.622 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout: ReadX 583182 0.094 2047.530 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout: LockX 1212 8.714 455.195 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout: UnlockX 1212 7.906 461.663 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout: Flush 26064 24.365 2787.025 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout: 2024-04-25T01:27:21.864 INFO:tasks.workunit.client.1.smithi171.stdout:Throughput 19.2613 MB/sec 10 clients 10 procs max_latency=2787.034 ms 2024-04-25T01:27:21.864 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:27:21.865 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-25T01:27:23.586 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.1... 2024-04-25T01:27:23.586 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-25T01:27:24.126 DEBUG:teuthology.parallel:result is None 2024-04-25T01:27:24.127 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T01:27:24.163 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T01:27:24.163 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T01:27:24.163 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:27:24.163 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:27:24.229 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T01:27:24.229 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:27:24.286 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:27:25.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.324+0000 7fc679b09640 1 Processor -- start 2024-04-25T01:27:25.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.324+0000 7fc679b09640 1 -- start start 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc679b09640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc67414f6f0 0x7fc67414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc679b09640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fc6740ac540 0x7fc6740c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc679b09640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6740c5180 con 0x7fc67406d6c0 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc679b09640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fc674155720 con 0x7fc6740ac540 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc679b09640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fc674153010 con 0x7fc67414f6f0 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc672ffd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fc6740ac540 0x7fc6740c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc673fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc67406d6c0 0x7fc6740c2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:33710/0 (socket says 172.21.15.129:33710) 2024-04-25T01:27:25.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc673fff640 1 -- 172.21.15.129:0/1881799509 learned_addr learned my addr 172.21.15.129:0/1881799509 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc6737fe640 1 --2- 172.21.15.129:0/1881799509 >> v2:172.21.15.171:3300/0 conn(0x7fc67414f6f0 0x7fc67414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc672ffd640 1 -- 172.21.15.129:0/1881799509 >> v2:172.21.15.171:3300/0 conn(0x7fc67414f6f0 msgr2=0x7fc67414fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc672ffd640 1 --2- 172.21.15.129:0/1881799509 >> v2:172.21.15.171:3300/0 conn(0x7fc67414f6f0 0x7fc67414fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc672ffd640 1 -- 172.21.15.129:0/1881799509 >> v1:172.21.15.129:6789/0 conn(0x7fc67406d6c0 legacy=0x7fc6740c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc6727fc640 1 -- 172.21.15.129:0/1881799509 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690687457 0 0) 0x7fc6740c5180 con 0x7fc67406d6c0 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.325+0000 7fc672ffd640 1 -- 172.21.15.129:0/1881799509 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6741559e0 con 0x7fc6740ac540 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.326+0000 7fc672ffd640 1 --2- 172.21.15.129:0/1881799509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6740c4800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc67406ec60 tx=0x7fc65c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c34b5002007073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.326+0000 7fc6727fc640 1 -- 172.21.15.129:0/1881799509 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c031710 con 0x7fc6740ac540 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.326+0000 7fc6727fc640 1 -- 172.21.15.129:0/1881799509 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc65c0318b0 con 0x7fc6740ac540 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.326+0000 7fc6727fc640 1 -- 172.21.15.129:0/1881799509 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c031c80 con 0x7fc6740ac540 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 -- 172.21.15.129:0/1881799509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 msgr2=0x7fc6740c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 --2- 172.21.15.129:0/1881799509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6740c4800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc67406ec60 tx=0x7fc65c02c7e0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 -- 172.21.15.129:0/1881799509 shutdown_connections 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 --2- 172.21.15.129:0/1881799509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6740c4800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 --2- 172.21.15.129:0/1881799509 >> v2:172.21.15.171:3300/0 conn(0x7fc67414f6f0 0x7fc67414fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 -- 172.21.15.129:0/1881799509 >> 172.21.15.129:0/1881799509 conn(0x7fc6740a86b0 msgr2=0x7fc6740a8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:25.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 -- 172.21.15.129:0/1881799509 shutdown_connections 2024-04-25T01:27:25.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.327+0000 7fc679b09640 1 -- 172.21.15.129:0/1881799509 wait complete. 2024-04-25T01:27:25.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.328+0000 7fc679b09640 1 Processor -- start 2024-04-25T01:27:25.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.328+0000 7fc679b09640 1 -- start start 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc67406d6c0 0x7fc6741eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6741512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc67414f6f0 0x7fc6740bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc674152c30 con 0x7fc6740ac540 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc674155fa0 con 0x7fc67406d6c0 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc674152db0 con 0x7fc67414f6f0 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6741512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6741512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60044/0 (socket says 172.21.15.129:60044) 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 -- 172.21.15.129:0/2228195465 learned_addr learned my addr 172.21.15.129:0/2228195465 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc6737fe640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc67406d6c0 0x7fc6741eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc673fff640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc67414f6f0 0x7fc6740bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 -- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc67414f6f0 msgr2=0x7fc6740bffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc67414f6f0 0x7fc6740bffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 -- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc67406d6c0 msgr2=0x7fc6741eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc67406d6c0 0x7fc6741eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 -- 172.21.15.129:0/2228195465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc674151d90 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc672ffd640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6741512d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc65c000c00 tx=0x7fc65c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c041070 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc65c033c60 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc674151810 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c006e10 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.329+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6741eae20 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.330+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6740c4b80 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.331+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc65c044070 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.331+0000 7fc66bfff640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc65406be30 0x7fc65406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.331+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc65c0abf10 con 0x7fc6740ac540 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.331+0000 7fc6737fe640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc65406be30 0x7fc65406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.332+0000 7fc6737fe640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc65406be30 0x7fc65406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6640113e0 tx=0x7fc66400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.339+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc6740c4b80 con 0x7fc6740ac540 2024-04-25T01:27:25.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.517+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc674075550 con 0x7fc6740ac540 2024-04-25T01:27:25.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.519+0000 7fc66bfff640 1 -- 172.21.15.129:0/2228195465 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18891 (secure 0 0 0) 0x7fc65c079f00 con 0x7fc6740ac540 2024-04-25T01:27:25.498 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:25.498 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:02:13.734125+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:27:25.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc65406be30 msgr2=0x7fc65406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc65406be30 0x7fc65406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6640113e0 tx=0x7fc66400b040 comp rx=0 tx=0).stop 2024-04-25T01:27:25.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 msgr2=0x7fc6741512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6741512d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc65c000c00 tx=0x7fc65c0046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 shutdown_connections 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc65406be30 0x7fc65406e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc67414f6f0 0x7fc6740bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6740ac540 0x7fc6741512d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 --2- 172.21.15.129:0/2228195465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc67406d6c0 0x7fc6741eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 >> 172.21.15.129:0/2228195465 conn(0x7fc6740a86b0 msgr2=0x7fc67414d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 shutdown_connections 2024-04-25T01:27:25.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.521+0000 7fc679b09640 1 -- 172.21.15.129:0/2228195465 wait complete. 2024-04-25T01:27:25.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:01:20.693388+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:01:30.640376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:01:31.706336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:01:35.424628+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:27:25.512 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:27:25.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.633+0000 7f4b8d96a640 1 Processor -- start 2024-04-25T01:27:25.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.633+0000 7f4b8d96a640 1 -- start start 2024-04-25T01:27:25.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b8d96a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f4b880c90a0 0x7f4b88159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b8d96a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c9f70 0x7f4b8815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b8d96a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b880c5c10 con 0x7f4b880c8cc0 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b8d96a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f4b880c9510 con 0x7f4b880c90a0 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b8d96a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880c5650 con 0x7f4b880c9f70 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b86ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f4b880c8cc0 0x7f4b88156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:33714/0 (socket says 172.21.15.129:33714) 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b86ffd640 1 -- 172.21.15.129:0/1553465298 learned_addr learned my addr 172.21.15.129:0/1553465298 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b867fc640 1 --2- 172.21.15.129:0/1553465298 >> v2:172.21.15.171:3300/0 conn(0x7f4b880c90a0 0x7f4b88159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b877fe640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c9f70 0x7f4b8815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b85ffb640 1 -- 172.21.15.129:0/1553465298 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917482624 0 0) 0x7f4b880c5c10 con 0x7f4b880c8cc0 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b85ffb640 1 -- 172.21.15.129:0/1553465298 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b74003520 con 0x7f4b880c8cc0 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b867fc640 1 -- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c9f70 msgr2=0x7f4b8815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b867fc640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c9f70 0x7f4b8815ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b867fc640 1 -- 172.21.15.129:0/1553465298 >> v1:172.21.15.129:6789/0 conn(0x7f4b880c8cc0 legacy=0x7f4b88156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b867fc640 1 -- 172.21.15.129:0/1553465298 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b880c5930 con 0x7f4b880c90a0 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.634+0000 7f4b877fe640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c9f70 0x7f4b8815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.635+0000 7f4b867fc640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c90a0 0x7f4b88159630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b780097a0 tx=0x7f4b7802a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef63d20e475beb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.635+0000 7f4b85ffb640 1 -- 172.21.15.129:0/1553465298 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78037070 con 0x7f4b880c90a0 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.635+0000 7f4b85ffb640 1 -- 172.21.15.129:0/1553465298 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7802ae40 con 0x7f4b880c90a0 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.635+0000 7f4b85ffb640 1 -- 172.21.15.129:0/1553465298 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78032660 con 0x7f4b880c90a0 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c90a0 msgr2=0x7f4b88159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c90a0 0x7f4b88159630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b780097a0 tx=0x7f4b7802a0f0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1553465298 shutdown_connections 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c9f70 0x7f4b8815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1553465298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c90a0 0x7f4b88159630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1553465298 >> 172.21.15.129:0/1553465298 conn(0x7f4b8806c400 msgr2=0x7f4b8806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:25.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1553465298 shutdown_connections 2024-04-25T01:27:25.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.636+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1553465298 wait complete. 2024-04-25T01:27:25.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.637+0000 7f4b8d96a640 1 Processor -- start 2024-04-25T01:27:25.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.637+0000 7f4b8d96a640 1 -- start start 2024-04-25T01:27:25.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 0x7f4b881e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c90a0 0x7f4b88143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 0x7f4b881441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8806d1f0 con 0x7f4b880c9f70 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880ca400 con 0x7f4b880c8cc0 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880c5f30 con 0x7f4b880c90a0 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 0x7f4b881441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 0x7f4b881e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b867fc640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c90a0 0x7f4b88143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 0x7f4b881441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60046/0 (socket says 172.21.15.129:60046) 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 -- 172.21.15.129:0/1604765870 learned_addr learned my addr 172.21.15.129:0/1604765870 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:25.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 0x7f4b881e25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:34358/0 (socket says 172.21.15.129:34358) 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 -- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c90a0 msgr2=0x7f4b88143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c90a0 0x7f4b88143cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 -- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 msgr2=0x7f4b881e25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 0x7f4b881e25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 -- 172.21.15.129:0/1604765870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88153cd0 con 0x7f4b880c9f70 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b86ffd640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 0x7f4b881e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b877fe640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 0x7f4b881441f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009e60 tx=0x7f4b8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80018030 con 0x7f4b880c9f70 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8000cdf0 con 0x7f4b880c9f70 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b88153fb0 con 0x7f4b880c9f70 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80003bb0 con 0x7f4b880c9f70 2024-04-25T01:27:25.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.638+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b88149b20 con 0x7f4b880c9f70 2024-04-25T01:27:25.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.639+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88075550 con 0x7f4b880c9f70 2024-04-25T01:27:25.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.640+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b8001b070 con 0x7f4b880c9f70 2024-04-25T01:27:25.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.640+0000 7f4b7ffff640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4b6806be30 0x7f4b6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.640+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4b8008bae0 con 0x7f4b880c9f70 2024-04-25T01:27:25.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.640+0000 7f4b86ffd640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4b6806be30 0x7f4b6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.641+0000 7f4b86ffd640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4b6806be30 0x7f4b6806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b70005880 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.648+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b80010040 con 0x7f4b880c9f70 2024-04-25T01:27:25.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.828+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b880bdf00 con 0x7f4b880c9f70 2024-04-25T01:27:25.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.829+0000 7f4b7ffff640 1 -- 172.21.15.129:0/1604765870 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f4b80059a40 con 0x7f4b880c9f70 2024-04-25T01:27:25.808 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-04-25T01:27:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4b6806be30 msgr2=0x7f4b6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4b6806be30 0x7f4b6806e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b70005880 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-04-25T01:27:25.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 msgr2=0x7f4b881441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 0x7f4b881441f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009e60 tx=0x7f4b8000c490 comp rx=0 tx=0).stop 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 shutdown_connections 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f4b6806be30 0x7f4b6806e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b880c90a0 0x7f4b88143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b880c9f70 0x7f4b881441f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 --2- 172.21.15.129:0/1604765870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b880c8cc0 0x7f4b881e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 >> 172.21.15.129:0/1604765870 conn(0x7f4b8806c400 msgr2=0x7f4b880ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 shutdown_connections 2024-04-25T01:27:25.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.831+0000 7f4b8d96a640 1 -- 172.21.15.129:0/1604765870 wait complete. 2024-04-25T01:27:25.821 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:27:25.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.944+0000 7f2253d42640 1 Processor -- start 2024-04-25T01:27:25.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.944+0000 7f2253d42640 1 -- start start 2024-04-25T01:27:25.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2253d42640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f224c0c8cc0 0x7f224c0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2253d42640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2253d42640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f224c156050 0x7f224c0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2253d42640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f224c0ca4b0 con 0x7f224c156050 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2253d42640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f224c0c5c10 con 0x7f224c0c8cc0 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2253d42640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0c5650 con 0x7f224c159630 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f224c156050 0x7f224c0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22512dd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2251ade640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f224c0c8cc0 0x7f224c0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f224c156050 0x7f224c0c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60048/0 (socket says 172.21.15.129:60048) 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 -- 172.21.15.129:0/451793985 learned_addr learned my addr 172.21.15.129:0/451793985 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 -- 172.21.15.129:0/451793985 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 msgr2=0x7f224c0c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 --2- 172.21.15.129:0/451793985 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 -- 172.21.15.129:0/451793985 >> v2:172.21.15.171:3300/0 conn(0x7f224c0c8cc0 msgr2=0x7f224c0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:25.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 --2- 172.21.15.129:0/451793985 >> v2:172.21.15.171:3300/0 conn(0x7f224c0c8cc0 0x7f224c0c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22522df640 1 -- 172.21.15.129:0/451793985 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c0c5930 con 0x7f224c156050 2024-04-25T01:27:25.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f2251ade640 1 --2- 172.21.15.129:0/451793985 >> v2:172.21.15.171:3300/0 conn(0x7f224c0c8cc0 0x7f224c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:25.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.945+0000 7f22512dd640 1 --2- 172.21.15.129:0/451793985 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.946+0000 7f22522df640 1 --2- 172.21.15.129:0/451793985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c156050 0x7f224c0c7580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ad7b0 tx=0x7f224402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a837f50cc294f4da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.947+0000 7f2250adc640 1 -- 172.21.15.129:0/451793985 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244036070 con 0x7f224c156050 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.947+0000 7f2250adc640 1 -- 172.21.15.129:0/451793985 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244031600 con 0x7f224c156050 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.947+0000 7f2250adc640 1 -- 172.21.15.129:0/451793985 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440319d0 con 0x7f224c156050 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 -- 172.21.15.129:0/451793985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c156050 msgr2=0x7f224c0c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 --2- 172.21.15.129:0/451793985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c156050 0x7f224c0c7580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ad7b0 tx=0x7f224402a610 comp rx=0 tx=0).stop 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 -- 172.21.15.129:0/451793985 shutdown_connections 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 --2- 172.21.15.129:0/451793985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c156050 0x7f224c0c7580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 --2- 172.21.15.129:0/451793985 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 --2- 172.21.15.129:0/451793985 >> v2:172.21.15.171:3300/0 conn(0x7f224c0c8cc0 0x7f224c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 -- 172.21.15.129:0/451793985 >> 172.21.15.129:0/451793985 conn(0x7f224c06c400 msgr2=0x7f224c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:25.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 -- 172.21.15.129:0/451793985 shutdown_connections 2024-04-25T01:27:25.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.948+0000 7f2253d42640 1 -- 172.21.15.129:0/451793985 wait complete. 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.949+0000 7f2253d42640 1 Processor -- start 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.949+0000 7f2253d42640 1 -- start start 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2253d42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 0x7f224c1ee750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2253d42640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f224c156050 0x7f224c0c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2253d42640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2253d42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f224c06d1f0 con 0x7f224c0c8cc0 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2253d42640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f224c159a70 con 0x7f224c156050 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2253d42640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0c5f30 con 0x7f224c159630 2024-04-25T01:27:25.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 0x7f224c1ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 0x7f224c1ee750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60062/0 (socket says 172.21.15.129:60062) 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 -- 172.21.15.129:0/2197366481 learned_addr learned my addr 172.21.15.129:0/2197366481 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f22522df640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f22512dd640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f224c156050 0x7f224c0c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 -- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 msgr2=0x7f224c0c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 -- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f224c156050 msgr2=0x7f224c0c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f224c156050 0x7f224c0c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 -- 172.21.15.129:0/2197366481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1ef420 con 0x7f224c0c8cc0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f2251ade640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 0x7f224c1ee750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f224800e910 tx=0x7f224800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.950+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248017070 con 0x7f224c0c8cc0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.951+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224800ca00 con 0x7f224c0c8cc0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.951+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c0c3bf0 con 0x7f224c0c8cc0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.951+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224800cdd0 con 0x7f224c0c8cc0 2024-04-25T01:27:25.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.951+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1ef140 con 0x7f224c0c8cc0 2024-04-25T01:27:25.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.952+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c075550 con 0x7f224c0c8cc0 2024-04-25T01:27:25.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.952+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f224801a070 con 0x7f224c0c8cc0 2024-04-25T01:27:25.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.953+0000 7f22427fc640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f223006be30 0x7f223006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:25.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.953+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f224808b2d0 con 0x7f224c0c8cc0 2024-04-25T01:27:25.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.953+0000 7f22512dd640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f223006be30 0x7f223006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:25.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.953+0000 7f22512dd640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f223006be30 0x7f223006e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f223c009560 tx=0x7f223c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:25.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:25.957+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22480592c0 con 0x7f224c0c8cc0 2024-04-25T01:27:26.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.137+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f224c0c9b40 con 0x7f224c0c8cc0 2024-04-25T01:27:26.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.138+0000 7f22427fc640 1 -- 172.21.15.129:0/2197366481 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f224808b0f0 con 0x7f224c0c8cc0 2024-04-25T01:27:26.117 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:27:26.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f223006be30 msgr2=0x7f223006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f223006be30 0x7f223006e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f223c009560 tx=0x7f223c009190 comp rx=0 tx=0).stop 2024-04-25T01:27:26.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 msgr2=0x7f224c1ee750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 0x7f224c1ee750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f224800e910 tx=0x7f224800edd0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 shutdown_connections 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f223006be30 0x7f223006e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f224c159630 0x7f224c0c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f224c156050 0x7f224c0c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 --2- 172.21.15.129:0/2197366481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f224c0c8cc0 0x7f224c1ee750 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 >> 172.21.15.129:0/2197366481 conn(0x7f224c06c400 msgr2=0x7f224c0ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 shutdown_connections 2024-04-25T01:27:26.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.140+0000 7f2253d42640 1 -- 172.21.15.129:0/2197366481 wait complete. 2024-04-25T01:27:26.130 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:27:26.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.264+0000 7f44cc363640 1 Processor -- start 2024-04-25T01:27:26.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.264+0000 7f44cc363640 1 -- start start 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44cc363640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c414e920 0x7f44c4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44cc363640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f44c40aa170 0x7f44c40abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44cc363640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c414ed00 con 0x7f44c40ac470 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44cc363640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f44c4152260 con 0x7f44c40aa170 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44cc363640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f44c40aaa80 con 0x7f44c414e920 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44ca900640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f44c40ac470 0x7f44c406c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:33716/0 (socket says 172.21.15.129:33716) 2024-04-25T01:27:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.265+0000 7f44ca900640 1 -- 172.21.15.129:0/2313069926 learned_addr learned my addr 172.21.15.129:0/2313069926 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:26.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.266+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562832732 0 0) 0x7f44c414ed00 con 0x7f44c40ac470 2024-04-25T01:27:26.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.266+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44b0003520 con 0x7f44c40ac470 2024-04-25T01:27:26.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1980284598 0 0) 0x7f44b0003520 con 0x7f44c40ac470 2024-04-25T01:27:26.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c414e920 msgr2=0x7f44c4151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:26.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c414e920 0x7f44c4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 >> v2:172.21.15.129:3300/0 conn(0x7f44c40aa170 msgr2=0x7f44c40abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 --2- 172.21.15.129:0/2313069926 >> v2:172.21.15.129:3300/0 conn(0x7f44c40aa170 0x7f44c40abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c406b8d0 con 0x7f44c40ac470 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f44bc0019f0 con 0x7f44c40ac470 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 >> v1:172.21.15.129:6789/0 conn(0x7f44c40ac470 legacy=0x7f44c406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44b0004800 0x7f44b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44b00052e0 0x7f44b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4152260 con 0x7f44b00039b0 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40aaa80 con 0x7f44b0004800 2024-04-25T01:27:26.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.269+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f44b000c1e0 con 0x7f44b00052e0 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.272+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.273+0000 7f44ca0ff640 1 -- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44b00052e0 msgr2=0x7f44b000b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.273+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44b00052e0 0x7f44b000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.273+0000 7f44ca0ff640 1 -- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44b0004800 msgr2=0x7f44b0004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.273+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44b0004800 0x7f44b0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.273+0000 7f44ca0ff640 1 -- 172.21.15.129:0/2313069926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c406b8d0 con 0x7f44b00039b0 2024-04-25T01:27:26.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.275+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44b0003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f44c40c7740 tx=0x7f44b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1325e78beeaa10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.284+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0037070 con 0x7f44b00039b0 2024-04-25T01:27:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.284+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c002a990 con 0x7f44b00039b0 2024-04-25T01:27:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.284+0000 7f44c90fd640 1 -- 172.21.15.129:0/2313069926 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c00093d0 con 0x7f44b00039b0 2024-04-25T01:27:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.288+0000 7f44cc363640 1 -- 172.21.15.129:0/2313069926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 msgr2=0x7f44b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.288+0000 7f44cc363640 1 --2- 172.21.15.129:0/2313069926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44b0003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f44c40c7740 tx=0x7f44b0005830 comp rx=0 tx=0).stop 2024-04-25T01:27:26.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.288+0000 7f44ca900640 1 -- 172.21.15.129:0/2313069926 reap_dead start 2024-04-25T01:27:26.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.295+0000 7f44cc363640 1 -- 172.21.15.129:0/2313069926 shutdown_connections 2024-04-25T01:27:26.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.295+0000 7f44cc363640 1 -- 172.21.15.129:0/2313069926 >> 172.21.15.129:0/2313069926 conn(0x7f44c40c3140 msgr2=0x7f44c40c3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.296+0000 7f44cc363640 1 -- 172.21.15.129:0/2313069926 shutdown_connections 2024-04-25T01:27:26.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.296+0000 7f44cc363640 1 -- 172.21.15.129:0/2313069926 wait complete. 2024-04-25T01:27:26.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.298+0000 7f44cc363640 1 Processor -- start 2024-04-25T01:27:26.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.298+0000 7f44cc363640 1 -- start start 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.299+0000 7f44cc363640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44c41e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.299+0000 7f44cc363640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44c40c0750 0x7f44c40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.299+0000 7f44cc363640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 0x7f44c4072650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.299+0000 7f44cc363640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44c406c170 con 0x7f44b00039b0 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.299+0000 7f44cc363640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f44c406be90 con 0x7f44c40c0750 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.299+0000 7f44cc363640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f44c406c450 con 0x7f44c4070f30 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.300+0000 7f44ca900640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 0x7f44c4072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.300+0000 7f44ca900640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 0x7f44c4072650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:37780/0 (socket says 172.21.15.129:37780) 2024-04-25T01:27:26.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.300+0000 7f44ca900640 1 -- 172.21.15.129:0/3066726266 learned_addr learned my addr 172.21.15.129:0/3066726266 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:26.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.300+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44c41e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.300+0000 7f44c98fe640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44c40c0750 0x7f44c40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.301+0000 7f44ca900640 1 -- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44c40c0750 msgr2=0x7f44c40709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.301+0000 7f44ca900640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44c40c0750 0x7f44c40709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.301+0000 7f44ca900640 1 -- 172.21.15.129:0/3066726266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 msgr2=0x7f44c41e73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.305+0000 7f44ca900640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44c41e73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.305+0000 7f44ca900640 1 -- 172.21.15.129:0/3066726266 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c41e7900 con 0x7f44c4070f30 2024-04-25T01:27:26.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.305+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44c41e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:26.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.316+0000 7f44ca900640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 0x7f44c4072650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ab0b0 tx=0x7f44bc00ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.320+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00a640 con 0x7f44c4070f30 2024-04-25T01:27:26.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.320+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c40bb870 con 0x7f44c4070f30 2024-04-25T01:27:26.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.320+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c4071b70 con 0x7f44c4070f30 2024-04-25T01:27:26.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.320+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc00a800 con 0x7f44c4070f30 2024-04-25T01:27:26.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.321+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00abf0 con 0x7f44c4070f30 2024-04-25T01:27:26.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.322+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac005290 con 0x7f44c4070f30 2024-04-25T01:27:26.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.323+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44bc007110 con 0x7f44c4070f30 2024-04-25T01:27:26.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.323+0000 7f44baffd640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f44a806bf00 0x7f44a806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.323+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f44bc08af90 con 0x7f44c4070f30 2024-04-25T01:27:26.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.328+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f44a806bf00 0x7f44a806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.331+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f44c41e7900 con 0x7f44c4070f30 2024-04-25T01:27:26.337 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.358+0000 7f44ca0ff640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f44a806bf00 0x7f44a806e3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44c0034000 tx=0x7f44c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.548+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f44ac005d90 con 0x7f44c4070f30 2024-04-25T01:27:26.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.548+0000 7f44baffd640 1 -- 172.21.15.129:0/3066726266 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f44bc0595b0 con 0x7f44c4070f30 2024-04-25T01:27:26.528 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f44a806bf00 msgr2=0x7f44a806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f44a806bf00 0x7f44a806e3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44c0034000 tx=0x7f44c002e000 comp rx=0 tx=0).stop 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 msgr2=0x7f44c4072650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 0x7f44c4072650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ab0b0 tx=0x7f44bc00ccd0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 shutdown_connections 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f44a806bf00 0x7f44a806e3c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44c4070f30 0x7f44c4072650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44c40c0750 0x7f44c40709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 --2- 172.21.15.129:0/3066726266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44b00039b0 0x7f44c41e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.550+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 >> 172.21.15.129:0/3066726266 conn(0x7f44c40c3140 msgr2=0x7f44c406c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.551+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 shutdown_connections 2024-04-25T01:27:26.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.551+0000 7f44cc363640 1 -- 172.21.15.129:0/3066726266 wait complete. 2024-04-25T01:27:26.540 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:27:26.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.662+0000 7f00c145d640 1 Processor -- start 2024-04-25T01:27:26.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.662+0000 7f00c145d640 1 -- start start 2024-04-25T01:27:26.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00c145d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f00bc0c8cc0 0x7f00bc0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00c145d640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00bc0c5c10 con 0x7f00bc0c9f70 2024-04-25T01:27:26.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00c145d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00bc159aa0 con 0x7f00bc159630 2024-04-25T01:27:26.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00c145d640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f00bc0c5650 con 0x7f00bc0c8cc0 2024-04-25T01:27:26.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00bb7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f00bc0c9f70 0x7f00bc15ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:33722/0 (socket says 172.21.15.129:33722) 2024-04-25T01:27:26.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00bb7fe640 1 -- 172.21.15.129:0/3933447176 learned_addr learned my addr 172.21.15.129:0/3933447176 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00baffd640 1 --2- 172.21.15.129:0/3933447176 >> v2:172.21.15.171:3300/0 conn(0x7f00bc0c8cc0 0x7f00bc0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00b9ffb640 1 -- 172.21.15.129:0/3933447176 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660684464 0 0) 0x7f00bc0c5c10 con 0x7f00bc0c9f70 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00b9ffb640 1 -- 172.21.15.129:0/3933447176 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a4003520 con 0x7f00bc0c9f70 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00baffd640 1 -- 172.21.15.129:0/3933447176 >> v1:172.21.15.171:6789/0 conn(0x7f00bc159630 legacy=0x7f00bc156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00baffd640 1 -- 172.21.15.129:0/3933447176 >> v1:172.21.15.129:6789/0 conn(0x7f00bc0c9f70 legacy=0x7f00bc15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.663+0000 7f00baffd640 1 -- 172.21.15.129:0/3933447176 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc0c5930 con 0x7f00bc0c8cc0 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.664+0000 7f00baffd640 1 --2- 172.21.15.129:0/3933447176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c90a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00b00097a0 tx=0x7f00b002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7bdc620794154ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.664+0000 7f00b9ffb640 1 -- 172.21.15.129:0/3933447176 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0037070 con 0x7f00bc0c8cc0 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.664+0000 7f00b9ffb640 1 -- 172.21.15.129:0/3933447176 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b0032440 con 0x7f00bc0c8cc0 2024-04-25T01:27:26.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.664+0000 7f00b9ffb640 1 -- 172.21.15.129:0/3933447176 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0032810 con 0x7f00bc0c8cc0 2024-04-25T01:27:26.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 -- 172.21.15.129:0/3933447176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 msgr2=0x7f00bc0c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 --2- 172.21.15.129:0/3933447176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c90a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00b00097a0 tx=0x7f00b002a1a0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 -- 172.21.15.129:0/3933447176 shutdown_connections 2024-04-25T01:27:26.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 --2- 172.21.15.129:0/3933447176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c90a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 -- 172.21.15.129:0/3933447176 >> 172.21.15.129:0/3933447176 conn(0x7f00bc06c400 msgr2=0x7f00bc06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 -- 172.21.15.129:0/3933447176 shutdown_connections 2024-04-25T01:27:26.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.665+0000 7f00c145d640 1 -- 172.21.15.129:0/3933447176 wait complete. 2024-04-25T01:27:26.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.666+0000 7f00c145d640 1 Processor -- start 2024-04-25T01:27:26.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.666+0000 7f00c145d640 1 -- start start 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00c145d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00c145d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 0x7f00bc157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00c145d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00bc159630 0x7f00bc1e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00c145d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc06d1f0 con 0x7f00bc0c9f70 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00c145d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc0ca3e0 con 0x7f00bc0c8cc0 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00c145d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc0c5f30 con 0x7f00bc159630 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 0x7f00bc157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 0x7f00bc157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60084/0 (socket says 172.21.15.129:60084) 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 -- 172.21.15.129:0/1580982358 learned_addr learned my addr 172.21.15.129:0/1580982358 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00baffd640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00bb7fe640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00bc159630 0x7f00bc1e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 -- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00bc159630 msgr2=0x7f00bc1e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00bc159630 0x7f00bc1e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 -- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 msgr2=0x7f00bc0c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 -- 172.21.15.129:0/1580982358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc14c7b0 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.667+0000 7f00ba7fc640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 0x7f00bc157ef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c430 tx=0x7f00ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.668+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0052d0 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.668+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac005490 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.668+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc0c16a0 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.668+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac007260 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.668+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1ec390 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.669+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc075550 con 0x7f00bc0c9f70 2024-04-25T01:27:26.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.669+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00ac007400 con 0x7f00bc0c9f70 2024-04-25T01:27:26.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.669+0000 7f00a37fe640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a806be30 0x7f00a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.669+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f00ac08a640 con 0x7f00bc0c9f70 2024-04-25T01:27:26.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.670+0000 7f00baffd640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a806be30 0x7f00a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.670+0000 7f00baffd640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a806be30 0x7f00a806e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00b0034000 tx=0x7f00b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.674+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00ac058630 con 0x7f00bc0c9f70 2024-04-25T01:27:26.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.855+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f00bc0c9b40 con 0x7f00bc0c9f70 2024-04-25T01:27:26.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.855+0000 7f00a37fe640 1 -- 172.21.15.129:0/1580982358 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f00ac05c930 con 0x7f00bc0c9f70 2024-04-25T01:27:26.835 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.857+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a806be30 msgr2=0x7f00a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.857+0000 7f00c145d640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a806be30 0x7f00a806e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00b0034000 tx=0x7f00b002e000 comp rx=0 tx=0).stop 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.857+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 msgr2=0x7f00bc157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.857+0000 7f00c145d640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 0x7f00bc157ef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c430 tx=0x7f00ac00c8f0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 shutdown_connections 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f00a806be30 0x7f00a806e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f00bc159630 0x7f00bc1e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00bc0c9f70 0x7f00bc157ef0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 --2- 172.21.15.129:0/1580982358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00bc0c8cc0 0x7f00bc0c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 >> 172.21.15.129:0/1580982358 conn(0x7f00bc06c400 msgr2=0x7f00bc06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 shutdown_connections 2024-04-25T01:27:26.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.858+0000 7f00c145d640 1 -- 172.21.15.129:0/1580982358 wait complete. 2024-04-25T01:27:26.848 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:27:26.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.968+0000 7f84460ba640 1 Processor -- start 2024-04-25T01:27:26.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.968+0000 7f84460ba640 1 -- start start 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f84460ba640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f8440152fa0 0x7f84400ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f84460ba640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84400c9860 con 0x7f84401561e0 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f84460ba640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84400c5a70 con 0x7f84400c5520 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f84460ba640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f84400c5d50 con 0x7f8440152fa0 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f843f7fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f8440152fa0 0x7f84400ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f843ffff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f84401561e0 0x7f84400c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:33724/0 (socket says 172.21.15.129:33724) 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.969+0000 7f843ffff640 1 -- 172.21.15.129:0/814527138 learned_addr learned my addr 172.21.15.129:0/814527138 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.970+0000 7f843f7fe640 1 -- 172.21.15.129:0/814527138 >> v1:172.21.15.171:6789/0 conn(0x7f84400c5520 legacy=0x7f84400c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.970+0000 7f843f7fe640 1 -- 172.21.15.129:0/814527138 >> v1:172.21.15.129:6789/0 conn(0x7f84401561e0 legacy=0x7f84400c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.970+0000 7f843e7fc640 1 -- 172.21.15.129:0/814527138 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350107600 0 0) 0x7f8438001110 con 0x7f84401561e0 2024-04-25T01:27:26.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.970+0000 7f843f7fe640 1 -- 172.21.15.129:0/814527138 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84400c7650 con 0x7f8440152fa0 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.971+0000 7f843f7fe640 1 --2- 172.21.15.129:0/814527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8440152fa0 0x7f84400ca2e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8430009d90 tx=0x7f843002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c542d17c38461b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.971+0000 7f843e7fc640 1 -- 172.21.15.129:0/814527138 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430031600 con 0x7f8440152fa0 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.971+0000 7f843e7fc640 1 -- 172.21.15.129:0/814527138 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84300317a0 con 0x7f8440152fa0 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.971+0000 7f843e7fc640 1 -- 172.21.15.129:0/814527138 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430031b70 con 0x7f8440152fa0 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 -- 172.21.15.129:0/814527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8440152fa0 msgr2=0x7f84400ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 --2- 172.21.15.129:0/814527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8440152fa0 0x7f84400ca2e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8430009d90 tx=0x7f843002a5d0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 -- 172.21.15.129:0/814527138 shutdown_connections 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 --2- 172.21.15.129:0/814527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8440152fa0 0x7f84400ca2e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 -- 172.21.15.129:0/814527138 >> 172.21.15.129:0/814527138 conn(0x7f844006c400 msgr2=0x7f844006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 -- 172.21.15.129:0/814527138 shutdown_connections 2024-04-25T01:27:26.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.972+0000 7f84460ba640 1 -- 172.21.15.129:0/814527138 wait complete. 2024-04-25T01:27:26.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 Processor -- start 2024-04-25T01:27:26.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 -- start start 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84400c5520 0x7f84400c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8440152fa0 0x7f84401ef820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 0x7f8440151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f84400c93d0 con 0x7f84401561e0 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84400c6030 con 0x7f8440152fa0 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.973+0000 7f84460ba640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f84400c9550 con 0x7f84400c5520 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 0x7f8440151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 0x7f8440151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60096/0 (socket says 172.21.15.129:60096) 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 -- 172.21.15.129:0/1550953722 learned_addr learned my addr 172.21.15.129:0/1550953722 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843f7fe640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84400c5520 0x7f84400c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843effd640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8440152fa0 0x7f84401ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 -- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84400c5520 msgr2=0x7f84400c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84400c5520 0x7f84400c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 -- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8440152fa0 msgr2=0x7f84401ef820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8440152fa0 0x7f84401ef820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 -- 172.21.15.129:0/1550953722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8440152480 con 0x7f84401561e0 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f843ffff640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 0x7f8440151630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f843800c780 tx=0x7f843800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438018070 con 0x7f84401561e0 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84380070d0 con 0x7f84401561e0 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401eb8a0 con 0x7f84401561e0 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438007500 con 0x7f84401561e0 2024-04-25T01:27:26.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.974+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8440158b90 con 0x7f84401561e0 2024-04-25T01:27:26.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.975+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440075550 con 0x7f84401561e0 2024-04-25T01:27:26.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.976+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f843801b070 con 0x7f84401561e0 2024-04-25T01:27:26.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.977+0000 7f8427fff640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f842c06be30 0x7f842c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:26.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.977+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f843808a3d0 con 0x7f84401561e0 2024-04-25T01:27:26.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.977+0000 7f843f7fe640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f842c06be30 0x7f842c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:26.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.977+0000 7f843f7fe640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f842c06be30 0x7f842c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8430034000 tx=0x7f843002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:26.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:26.981+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f84400c9860 con 0x7f84401561e0 2024-04-25T01:27:27.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.188+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8440156880 con 0x7f84401561e0 2024-04-25T01:27:27.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.188+0000 7f8427fff640 1 -- 172.21.15.129:0/1550953722 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f84380583c0 con 0x7f84401561e0 2024-04-25T01:27:27.167 INFO:teuthology.orchestra.run.smithi129.stderr:listed 0 entries 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f842c06be30 msgr2=0x7f842c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f842c06be30 0x7f842c06e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8430034000 tx=0x7f843002f000 comp rx=0 tx=0).stop 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 msgr2=0x7f8440151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 0x7f8440151630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f843800c780 tx=0x7f843800cc40 comp rx=0 tx=0).stop 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 shutdown_connections 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f842c06be30 0x7f842c06e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f84401561e0 0x7f8440151630 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8440152fa0 0x7f84401ef820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 --2- 172.21.15.129:0/1550953722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84400c5520 0x7f84400c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 >> 172.21.15.129:0/1550953722 conn(0x7f844006c400 msgr2=0x7f844006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 shutdown_connections 2024-04-25T01:27:27.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:27.190+0000 7f84460ba640 1 -- 172.21.15.129:0/1550953722 wait complete. 2024-04-25T01:27:27.181 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T01:27:27.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:27:27.181 DEBUG:teuthology.orchestra.run.smithi171:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T01:27:28.184 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:27:28.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:27:28.184 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T01:27:28.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:27:28.219 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-04-25T01:27:28.296 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T01:27:28.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:27:28.296 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:27:28.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:27:28.390 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-04-25T01:27:28.486 DEBUG:teuthology.parallel:result is None 2024-04-25T01:27:28.486 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T01:27:28.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T01:27:28.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:27:28.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.681+0000 7f05a2314640 1 Processor -- start 2024-04-25T01:27:28.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.681+0000 7f05a2314640 1 -- start start 2024-04-25T01:27:28.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.682+0000 7f05a2314640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f059c070e10 0x7f059c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.683+0000 7f05a08b1640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f059c0c6eb0 0x7f059c15da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60098/0 (socket says 172.21.15.129:60098) 2024-04-25T01:27:28.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.683+0000 7f05a08b1640 1 -- 172.21.15.129:0/2639343328 learned_addr learned my addr 172.21.15.129:0/2639343328 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:28.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.683+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f059c153da0 con 0x7f059c0c6eb0 2024-04-25T01:27:28.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.683+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535656629 0 0) 0x7f059c153da0 con 0x7f059c0c6eb0 2024-04-25T01:27:28.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.683+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f059c0c8350 con 0x7f059c0c4e50 2024-04-25T01:27:28.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.684+0000 7f059bfff640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f059c070e10 0x7f059c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:28.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.684+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f059c0c6990 con 0x7f059c070e10 2024-04-25T01:27:28.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0584003520 con 0x7f059c0c6eb0 2024-04-25T01:27:28.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496645312 0 0) 0x7f059c0c8350 con 0x7f059c0c4e50 2024-04-25T01:27:28.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f059c153da0 con 0x7f059c0c4e50 2024-04-25T01:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 785118630 0 0) 0x7f0584003520 con 0x7f059c0c6eb0 2024-04-25T01:27:28.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f059c070e10 msgr2=0x7f059c06e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f059c070e10 0x7f059c06e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 >> v1:172.21.15.171:6789/0 conn(0x7f059c0c4e50 legacy=0x7f059c0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c153ab0 con 0x7f059c0c6eb0 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f05940019f0 con 0x7f059c0c6eb0 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 >> v1:172.21.15.129:6789/0 conn(0x7f059c0c6eb0 legacy=0x7f059c15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0584004bd0 0x7f0584004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05840037a0 0x7f0584003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0584004160 0x7f058400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f059c0c6990 con 0x7f05840037a0 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f058400c090 con 0x7f0584004bd0 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.686+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f058400c260 con 0x7f0584004160 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05840037a0 0x7f0584003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:28.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f05a08b1640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0584004160 0x7f058400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059bfff640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0584004bd0 0x7f0584004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059b7fe640 1 -- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0584004160 msgr2=0x7f058400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0584004160 0x7f058400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059b7fe640 1 -- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0584004bd0 msgr2=0x7f0584004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0584004bd0 0x7f0584004fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.687+0000 7f059b7fe640 1 -- 172.21.15.129:0/2639343328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c153ab0 con 0x7f05840037a0 2024-04-25T01:27:28.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.688+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05840037a0 0x7f0584003c20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f05900018a0 tx=0x7f0584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d75e490380b17ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.689+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900014c0 con 0x7f05840037a0 2024-04-25T01:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.689+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059002e070 con 0x7f05840037a0 2024-04-25T01:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.689+0000 7f059affd640 1 -- 172.21.15.129:0/2639343328 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590005070 con 0x7f05840037a0 2024-04-25T01:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.690+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05840037a0 msgr2=0x7f0584003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.690+0000 7f05a2314640 1 --2- 172.21.15.129:0/2639343328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05840037a0 0x7f0584003c20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f05900018a0 tx=0x7f0584005830 comp rx=0 tx=0).stop 2024-04-25T01:27:28.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.690+0000 7f05a08b1640 1 -- 172.21.15.129:0/2639343328 reap_dead start 2024-04-25T01:27:28.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.692+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 shutdown_connections 2024-04-25T01:27:28.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.692+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 >> 172.21.15.129:0/2639343328 conn(0x7f059c0c5490 msgr2=0x7f059c0c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:28.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.696+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 shutdown_connections 2024-04-25T01:27:28.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.696+0000 7f05a2314640 1 -- 172.21.15.129:0/2639343328 wait complete. 2024-04-25T01:27:28.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.698+0000 7f05a2314640 1 Processor -- start 2024-04-25T01:27:28.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.700+0000 7f05a2314640 1 -- start start 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f05a2314640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05840037a0 0x7f059c0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f05a2314640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 0x7f059c149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f05a2314640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f059c0a86b0 0x7f059c147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f05a2314640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f059c0aade0 con 0x7f059c1e9b50 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f05a2314640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f059c0c8630 con 0x7f059c0a86b0 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f05a2314640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f059c0aaf60 con 0x7f05840037a0 2024-04-25T01:27:28.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f059b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 0x7f059c149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f059b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 0x7f059c149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55312/0 (socket says 172.21.15.129:55312) 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.704+0000 7f059b7fe640 1 -- 172.21.15.129:0/2631828447 learned_addr learned my addr 172.21.15.129:0/2631828447 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.705+0000 7f059b7fe640 1 -- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05840037a0 msgr2=0x7f059c0adcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.705+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05840037a0 0x7f059c0adcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.705+0000 7f059b7fe640 1 -- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f059c0a86b0 msgr2=0x7f059c147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.705+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f059c0a86b0 0x7f059c147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.705+0000 7f059b7fe640 1 -- 172.21.15.129:0/2631828447 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c0acc10 con 0x7f059c1e9b50 2024-04-25T01:27:28.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.705+0000 7f059b7fe640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 0x7f059c149120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05900019c0 tx=0x7f05900012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:28.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.706+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590043070 con 0x7f059c1e9b50 2024-04-25T01:27:28.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.706+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590003db0 con 0x7f059c1e9b50 2024-04-25T01:27:28.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.707+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900014c0 con 0x7f059c1e9b50 2024-04-25T01:27:28.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.708+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c0acef0 con 0x7f059c1e9b50 2024-04-25T01:27:28.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.708+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c0ad4b0 con 0x7f059c1e9b50 2024-04-25T01:27:28.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.709+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059c075550 con 0x7f059c1e9b50 2024-04-25T01:27:28.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.710+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f059003e670 con 0x7f059c1e9b50 2024-04-25T01:27:28.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.711+0000 7f0598ff9640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f058806beb0 0x7f058806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:28.695 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.716+0000 7f059bfff640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f058806beb0 0x7f058806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:28.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.716+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0590076cc0 con 0x7f059c1e9b50 2024-04-25T01:27:28.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.720+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f05900b1030 con 0x7f059c1e9b50 2024-04-25T01:27:28.705 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:28.726+0000 7f059bfff640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f058806beb0 0x7f058806e370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f058c006fd0 tx=0x7f058c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.107+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f059c150680 con 0x7f058806beb0 2024-04-25T01:27:29.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.123+0000 7f0598ff9640 1 -- 172.21.15.129:0/2631828447 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336161 (secure 0 0 0) 0x7f059c150680 con 0x7f058806beb0 2024-04-25T01:27:29.107 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:29.110 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:27:29.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f058806beb0 msgr2=0x7f058806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f058806beb0 0x7f058806e370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f058c006fd0 tx=0x7f058c008040 comp rx=0 tx=0).stop 2024-04-25T01:27:29.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 msgr2=0x7f059c149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 0x7f059c149120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05900019c0 tx=0x7f05900012d0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 shutdown_connections 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f058806beb0 0x7f058806e370 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f059c0a86b0 0x7f059c147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f059c1e9b50 0x7f059c149120 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05900019c0 tx=0x7f05900012d0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 --2- 172.21.15.129:0/2631828447 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f05840037a0 0x7f059c0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.128+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 >> 172.21.15.129:0/2631828447 conn(0x7f059c0c5490 msgr2=0x7f059c06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.130+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 shutdown_connections 2024-04-25T01:27:29.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.130+0000 7f05a2314640 1 -- 172.21.15.129:0/2631828447 wait complete. 2024-04-25T01:27:29.128 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":805,"stamp":"2024-04-25T01:27:29.057652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":309559771,"num_objects":602,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":602,"num_whiteouts":0,"num_read":15246,"num_read_kb":3072246,"num_write":422204,"num_write_kb":32347863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":70,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588086,"ondisk_log_size":588086,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8133900,"kb_used_data":608076,"kb_used_omap":12,"kb_used_meta":7525747,"kb_avail":746840820,"statfs":{"total":773094113280,"available":764764999680,"internally_reserved":0,"allocated":622669824,"data_stored":621228160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7706365488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-172834971,"num_objects":-1095,"num_object_clones":0,"num_object_copies":-3404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1095,"num_whiteouts":0,"num_read":40,"num_read_kb":6703,"num_write":3385,"num_write_kb":53731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002804"},"pg_stats":[{"pgid":"2.1f","version":"27'2272","reported_seq":2310,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:35.725226+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:26:35.725226+0000","last_peered":"2024-04-25T01:26:35.725226+0000","last_clean":"2024-04-25T01:26:35.725226+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:26:35.725226+0000","last_undegraded":"2024-04-25T01:26:35.725226+0000","last_fullsized":"2024-04-25T01:26:35.725226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2281,"num_write_kb":17945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2538","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.357273+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:27:23.357273+0000","last_peered":"2024-04-25T01:27:23.357273+0000","last_clean":"2024-04-25T01:27:23.357273+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:27:23.357273+0000","last_undegraded":"2024-04-25T01:27:23.357273+0000","last_fullsized":"2024-04-25T01:27:23.357273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2538,"log_dups_size":0,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9425","reported_seq":9743,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.300308+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:27:23.300308+0000","last_peered":"2024-04-25T01:27:23.300308+0000","last_clean":"2024-04-25T01:27:23.300308+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:27:23.300308+0000","last_undegraded":"2024-04-25T01:27:23.300308+0000","last_fullsized":"2024-04-25T01:27:23.300308+0000","mapping_epoch":23,"log_start":"27'3102","ondisk_log_start":"27'3102","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754914,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":290,"num_read_kb":73660,"num_write":6789,"num_write_kb":1006904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6102","reported_seq":6140,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:00.225983+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:27:00.225983+0000","last_peered":"2024-04-25T01:27:00.225983+0000","last_clean":"2024-04-25T01:27:00.225983+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:27:00.225983+0000","last_undegraded":"2024-04-25T01:27:00.225983+0000","last_fullsized":"2024-04-25T01:27:00.225983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6102,"log_dups_size":0,"ondisk_log_size":6102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6110,"num_write_kb":49190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2508","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.936451+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:27:22.936451+0000","last_peered":"2024-04-25T01:27:22.936451+0000","last_clean":"2024-04-25T01:27:22.936451+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:27:22.936451+0000","last_undegraded":"2024-04-25T01:27:22.936451+0000","last_fullsized":"2024-04-25T01:27:22.936451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9551","reported_seq":10008,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.674548+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:27:27.674548+0000","last_peered":"2024-04-25T01:27:27.674548+0000","last_clean":"2024-04-25T01:27:27.674548+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:27:27.674548+0000","last_undegraded":"2024-04-25T01:27:27.674548+0000","last_fullsized":"2024-04-25T01:27:27.674548+0000","mapping_epoch":23,"log_start":"27'2331","ondisk_log_start":"27'2331","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2241862,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":429,"num_read_kb":108966,"num_write":6803,"num_write_kb":1087428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9177","reported_seq":9304,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.354011+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:27:26.354011+0000","last_peered":"2024-04-25T01:27:26.354011+0000","last_clean":"2024-04-25T01:27:26.354011+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:27:26.354011+0000","last_undegraded":"2024-04-25T01:27:26.354011+0000","last_fullsized":"2024-04-25T01:27:26.354011+0000","mapping_epoch":18,"log_start":"27'2200","ondisk_log_start":"27'2200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6977,"log_dups_size":0,"ondisk_log_size":6977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4929315,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":772,"num_write":9170,"num_write_kb":76431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2627","reported_seq":2661,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.334468+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:27:28.334468+0000","last_peered":"2024-04-25T01:27:28.334468+0000","last_clean":"2024-04-25T01:27:28.334468+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:27:28.334468+0000","last_undegraded":"2024-04-25T01:27:28.334468+0000","last_fullsized":"2024-04-25T01:27:28.334468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2627,"log_dups_size":0,"ondisk_log_size":2627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9134","reported_seq":9750,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.567004+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:27:27.567004+0000","last_peered":"2024-04-25T01:27:27.567004+0000","last_clean":"2024-04-25T01:27:27.567004+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:27:27.567004+0000","last_undegraded":"2024-04-25T01:27:27.567004+0000","last_fullsized":"2024-04-25T01:27:27.567004+0000","mapping_epoch":23,"log_start":"27'1914","ondisk_log_start":"27'1914","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3266670,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6424,"num_write_kb":913615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4331","reported_seq":4367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.374958+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:27:26.374958+0000","last_peered":"2024-04-25T01:27:26.374958+0000","last_clean":"2024-04-25T01:27:26.374958+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:27:26.374958+0000","last_undegraded":"2024-04-25T01:27:26.374958+0000","last_fullsized":"2024-04-25T01:27:26.374958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4333,"num_write_kb":43749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2555","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.310340+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:27:28.310340+0000","last_peered":"2024-04-25T01:27:28.310340+0000","last_clean":"2024-04-25T01:27:28.310340+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:27:28.310340+0000","last_undegraded":"2024-04-25T01:27:28.310340+0000","last_fullsized":"2024-04-25T01:27:28.310340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8828","reported_seq":9239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.693327+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:27:27.693327+0000","last_peered":"2024-04-25T01:27:27.693327+0000","last_clean":"2024-04-25T01:27:27.693327+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:27:27.693327+0000","last_undegraded":"2024-04-25T01:27:27.693327+0000","last_fullsized":"2024-04-25T01:27:27.693327+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1257244,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":383,"num_read_kb":97282,"num_write":6126,"num_write_kb":837257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2548","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.095129+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:27:23.095129+0000","last_peered":"2024-04-25T01:27:23.095129+0000","last_clean":"2024-04-25T01:27:23.095129+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:27:23.095129+0000","last_undegraded":"2024-04-25T01:27:23.095129+0000","last_fullsized":"2024-04-25T01:27:23.095129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6999","reported_seq":7036,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.366224+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:27:26.366224+0000","last_peered":"2024-04-25T01:27:26.366224+0000","last_clean":"2024-04-25T01:27:26.366224+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:27:26.366224+0000","last_undegraded":"2024-04-25T01:27:26.366224+0000","last_fullsized":"2024-04-25T01:27:26.366224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6999,"log_dups_size":0,"ondisk_log_size":6999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6999,"num_write_kb":53860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9950","reported_seq":10572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.681856+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:27:27.681856+0000","last_peered":"2024-04-25T01:27:27.681856+0000","last_clean":"2024-04-25T01:27:27.681856+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:27:27.681856+0000","last_undegraded":"2024-04-25T01:27:27.681856+0000","last_fullsized":"2024-04-25T01:27:27.681856+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1888996,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":593,"num_read_kb":150622,"num_write":7196,"num_write_kb":1017659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2562","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.956703+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:27:22.956703+0000","last_peered":"2024-04-25T01:27:22.956703+0000","last_clean":"2024-04-25T01:27:22.956703+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:27:22.956703+0000","last_undegraded":"2024-04-25T01:27:22.956703+0000","last_fullsized":"2024-04-25T01:27:22.956703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7175","reported_seq":7212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.389762+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:27:26.389762+0000","last_peered":"2024-04-25T01:27:26.389762+0000","last_clean":"2024-04-25T01:27:26.389762+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:27:26.389762+0000","last_undegraded":"2024-04-25T01:27:26.389762+0000","last_fullsized":"2024-04-25T01:27:26.389762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7178,"num_write_kb":58045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9175","reported_seq":9667,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.692326+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:27:27.692326+0000","last_peered":"2024-04-25T01:27:27.692326+0000","last_clean":"2024-04-25T01:27:27.692326+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:27:27.692326+0000","last_undegraded":"2024-04-25T01:27:27.692326+0000","last_fullsized":"2024-04-25T01:27:27.692326+0000","mapping_epoch":23,"log_start":"27'1927","ondisk_log_start":"27'1927","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7248,"log_dups_size":0,"ondisk_log_size":7248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1152186,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":467,"num_read_kb":118618,"num_write":6473,"num_write_kb":966622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2628","reported_seq":2663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.092732+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:27:23.092732+0000","last_peered":"2024-04-25T01:27:23.092732+0000","last_clean":"2024-04-25T01:27:23.092732+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:27:23.092732+0000","last_undegraded":"2024-04-25T01:27:23.092732+0000","last_fullsized":"2024-04-25T01:27:23.092732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2628,"log_dups_size":0,"ondisk_log_size":2628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3548","reported_seq":3585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.339055+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:27:26.339055+0000","last_peered":"2024-04-25T01:27:26.339055+0000","last_clean":"2024-04-25T01:27:26.339055+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:27:26.339055+0000","last_undegraded":"2024-04-25T01:27:26.339055+0000","last_fullsized":"2024-04-25T01:27:26.339055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":26965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'8958","reported_seq":9474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.403662+0000","last_change":"2024-04-25T01:02:13.748397+0000","last_active":"2024-04-25T01:27:23.403662+0000","last_peered":"2024-04-25T01:27:23.403662+0000","last_clean":"2024-04-25T01:27:23.403662+0000","last_became_active":"2024-04-25T01:02:13.748037+0000","last_became_peered":"2024-04-25T01:02:13.748037+0000","last_unstale":"2024-04-25T01:27:23.403662+0000","last_undegraded":"2024-04-25T01:27:23.403662+0000","last_fullsized":"2024-04-25T01:27:23.403662+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":8958,"log_dups_size":0,"ondisk_log_size":8958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":616768,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6393,"num_write_kb":892388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2527","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.249690+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:27:23.249690+0000","last_peered":"2024-04-25T01:27:23.249690+0000","last_clean":"2024-04-25T01:27:23.249690+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:27:23.249690+0000","last_undegraded":"2024-04-25T01:27:23.249690+0000","last_fullsized":"2024-04-25T01:27:23.249690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2989","reported_seq":3026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.361419+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:27:26.361419+0000","last_peered":"2024-04-25T01:27:26.361419+0000","last_clean":"2024-04-25T01:27:26.361419+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:27:26.361419+0000","last_undegraded":"2024-04-25T01:27:26.361419+0000","last_fullsized":"2024-04-25T01:27:26.361419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2989,"log_dups_size":0,"ondisk_log_size":2989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2993,"num_write_kb":22443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9521","reported_seq":9738,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.674466+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:27:27.674466+0000","last_peered":"2024-04-25T01:27:27.674466+0000","last_clean":"2024-04-25T01:27:27.674466+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:27:27.674466+0000","last_undegraded":"2024-04-25T01:27:27.674466+0000","last_fullsized":"2024-04-25T01:27:27.674466+0000","mapping_epoch":23,"log_start":"27'2346","ondisk_log_start":"27'2346","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839412,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":193,"num_read_kb":49022,"num_write":6885,"num_write_kb":954026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2491","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.998935+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:27:22.998935+0000","last_peered":"2024-04-25T01:27:22.998935+0000","last_clean":"2024-04-25T01:27:22.998935+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:27:22.998935+0000","last_undegraded":"2024-04-25T01:27:22.998935+0000","last_fullsized":"2024-04-25T01:27:22.998935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2491,"log_dups_size":0,"ondisk_log_size":2491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4621","reported_seq":4658,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:40.881225+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:26:40.881225+0000","last_peered":"2024-04-25T01:26:40.881225+0000","last_clean":"2024-04-25T01:26:40.881225+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:26:40.881225+0000","last_undegraded":"2024-04-25T01:26:40.881225+0000","last_fullsized":"2024-04-25T01:26:40.881225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":39981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9238","reported_seq":9305,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.523276+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:27:27.523276+0000","last_peered":"2024-04-25T01:27:27.523276+0000","last_clean":"2024-04-25T01:27:27.523276+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:27:27.523276+0000","last_undegraded":"2024-04-25T01:27:27.523276+0000","last_fullsized":"2024-04-25T01:27:27.523276+0000","mapping_epoch":23,"log_start":"27'2005","ondisk_log_start":"27'2005","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820727,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":37,"num_read_kb":9398,"num_write":6593,"num_write_kb":862411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2484","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.960479+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:27:22.960479+0000","last_peered":"2024-04-25T01:27:22.960479+0000","last_clean":"2024-04-25T01:27:22.960479+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:27:22.960479+0000","last_undegraded":"2024-04-25T01:27:22.960479+0000","last_fullsized":"2024-04-25T01:27:22.960479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2484,"log_dups_size":0,"ondisk_log_size":2484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6901","reported_seq":7601,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.390267+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:27:26.390267+0000","last_peered":"2024-04-25T01:27:26.390267+0000","last_clean":"2024-04-25T01:27:26.390267+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:27:26.390267+0000","last_undegraded":"2024-04-25T01:27:26.390267+0000","last_fullsized":"2024-04-25T01:27:26.390267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":663,"num_read_kb":4517,"num_write":9702,"num_write_kb":61055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8977","reported_seq":9049,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.399133+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:27:23.399133+0000","last_peered":"2024-04-25T01:27:23.399133+0000","last_clean":"2024-04-25T01:27:23.399133+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:27:23.399133+0000","last_undegraded":"2024-04-25T01:27:23.399133+0000","last_fullsized":"2024-04-25T01:27:23.399133+0000","mapping_epoch":23,"log_start":"27'2722","ondisk_log_start":"27'2722","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2121035,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":10668,"num_write":6308,"num_write_kb":892747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2580","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.405166+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:27:23.405166+0000","last_peered":"2024-04-25T01:27:23.405166+0000","last_clean":"2024-04-25T01:27:23.405166+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:27:23.405166+0000","last_undegraded":"2024-04-25T01:27:23.405166+0000","last_fullsized":"2024-04-25T01:27:23.405166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6064","reported_seq":6102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322120+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:27:26.322120+0000","last_peered":"2024-04-25T01:27:26.322120+0000","last_clean":"2024-04-25T01:27:26.322120+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:27:26.322120+0000","last_undegraded":"2024-04-25T01:27:26.322120+0000","last_fullsized":"2024-04-25T01:27:26.322120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6062,"num_write_kb":49091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9783","reported_seq":10295,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.349957+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:27:23.349957+0000","last_peered":"2024-04-25T01:27:23.349957+0000","last_clean":"2024-04-25T01:27:23.349957+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:27:23.349957+0000","last_undegraded":"2024-04-25T01:27:23.349957+0000","last_fullsized":"2024-04-25T01:27:23.349957+0000","mapping_epoch":23,"log_start":"27'2453","ondisk_log_start":"27'2453","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2380367,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":482,"num_read_kb":122428,"num_write":7046,"num_write_kb":849139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2499","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.314796+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:27:28.314796+0000","last_peered":"2024-04-25T01:27:28.314796+0000","last_clean":"2024-04-25T01:27:28.314796+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:27:28.314796+0000","last_undegraded":"2024-04-25T01:27:28.314796+0000","last_fullsized":"2024-04-25T01:27:28.314796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7885","reported_seq":7923,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:11.111824+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:27:11.111824+0000","last_peered":"2024-04-25T01:27:11.111824+0000","last_clean":"2024-04-25T01:27:11.111824+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:27:11.111824+0000","last_undegraded":"2024-04-25T01:27:11.111824+0000","last_fullsized":"2024-04-25T01:27:11.111824+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7873,"num_write_kb":70555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9266","reported_seq":9744,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.927730+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:27:27.927730+0000","last_peered":"2024-04-25T01:27:27.927730+0000","last_clean":"2024-04-25T01:27:27.927730+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:27:27.927730+0000","last_undegraded":"2024-04-25T01:27:27.927730+0000","last_fullsized":"2024-04-25T01:27:27.927730+0000","mapping_epoch":23,"log_start":"27'1900","ondisk_log_start":"27'1900","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2553648,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":460,"num_read_kb":116840,"num_write":6586,"num_write_kb":921401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2522","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.480449+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:27:23.480449+0000","last_peered":"2024-04-25T01:27:23.480449+0000","last_clean":"2024-04-25T01:27:23.480449+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:27:23.480449+0000","last_undegraded":"2024-04-25T01:27:23.480449+0000","last_fullsized":"2024-04-25T01:27:23.480449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8860","reported_seq":8898,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.115869+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:27:28.115869+0000","last_peered":"2024-04-25T01:27:28.115869+0000","last_clean":"2024-04-25T01:27:28.115869+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:27:28.115869+0000","last_undegraded":"2024-04-25T01:27:28.115869+0000","last_fullsized":"2024-04-25T01:27:28.115869+0000","mapping_epoch":18,"log_start":"27'1500","ondisk_log_start":"27'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7360,"log_dups_size":0,"ondisk_log_size":7360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7300960,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8857,"num_write_kb":62366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9899","reported_seq":10045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.832406+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:27:27.832406+0000","last_peered":"2024-04-25T01:27:27.832406+0000","last_clean":"2024-04-25T01:27:27.832406+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:27:27.832406+0000","last_undegraded":"2024-04-25T01:27:27.832406+0000","last_fullsized":"2024-04-25T01:27:27.832406+0000","mapping_epoch":23,"log_start":"27'2523","ondisk_log_start":"27'2523","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7376,"log_dups_size":0,"ondisk_log_size":7376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071863,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":124,"num_read_kb":31496,"num_write":7205,"num_write_kb":913227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.249751+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:27:23.249751+0000","last_peered":"2024-04-25T01:27:23.249751+0000","last_clean":"2024-04-25T01:27:23.249751+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:27:23.249751+0000","last_undegraded":"2024-04-25T01:27:23.249751+0000","last_fullsized":"2024-04-25T01:27:23.249751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7133","reported_seq":8083,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:00.219862+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:27:00.219862+0000","last_peered":"2024-04-25T01:27:00.219862+0000","last_clean":"2024-04-25T01:27:00.219862+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:27:00.219862+0000","last_undegraded":"2024-04-25T01:27:00.219862+0000","last_fullsized":"2024-04-25T01:27:00.219862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7133,"log_dups_size":0,"ondisk_log_size":7133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":912,"num_read_kb":4677,"num_write":7121,"num_write_kb":61966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9048","reported_seq":9471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.972691+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:27:27.972691+0000","last_peered":"2024-04-25T01:27:27.972691+0000","last_clean":"2024-04-25T01:27:27.972691+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:27:27.972691+0000","last_undegraded":"2024-04-25T01:27:27.972691+0000","last_fullsized":"2024-04-25T01:27:27.972691+0000","mapping_epoch":23,"log_start":"27'1700","ondisk_log_start":"27'1700","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7348,"log_dups_size":0,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4129696,"num_objects":18,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":396,"num_read_kb":100584,"num_write":6310,"num_write_kb":903694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2472","reported_seq":2508,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.104533+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:27:23.104533+0000","last_peered":"2024-04-25T01:27:23.104533+0000","last_clean":"2024-04-25T01:27:23.104533+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:27:23.104533+0000","last_undegraded":"2024-04-25T01:27:23.104533+0000","last_fullsized":"2024-04-25T01:27:23.104533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'6806","reported_seq":6844,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.391028+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:27:26.391028+0000","last_peered":"2024-04-25T01:27:26.391028+0000","last_clean":"2024-04-25T01:27:26.391028+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:27:26.391028+0000","last_undegraded":"2024-04-25T01:27:26.391028+0000","last_fullsized":"2024-04-25T01:27:26.391028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6806,"num_write_kb":57773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9170","reported_seq":9528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.406550+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:27:23.406550+0000","last_peered":"2024-04-25T01:27:23.406550+0000","last_clean":"2024-04-25T01:27:23.406550+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:27:23.406550+0000","last_undegraded":"2024-04-25T01:27:23.406550+0000","last_fullsized":"2024-04-25T01:27:23.406550+0000","mapping_epoch":23,"log_start":"27'1826","ondisk_log_start":"27'1826","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3481570,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":329,"num_read_kb":83566,"num_write":6508,"num_write_kb":1104508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2578","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.129101+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:27:23.129101+0000","last_peered":"2024-04-25T01:27:23.129101+0000","last_clean":"2024-04-25T01:27:23.129101+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:27:23.129101+0000","last_undegraded":"2024-04-25T01:27:23.129101+0000","last_fullsized":"2024-04-25T01:27:23.129101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4347","reported_seq":4385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.334344+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:27:26.334344+0000","last_peered":"2024-04-25T01:27:26.334344+0000","last_clean":"2024-04-25T01:27:26.334344+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:27:26.334344+0000","last_undegraded":"2024-04-25T01:27:26.334344+0000","last_fullsized":"2024-04-25T01:27:26.334344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4347,"num_write_kb":39756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:23:30.725659+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:23:30.725659+0000","last_peered":"2024-04-25T01:23:30.725659+0000","last_clean":"2024-04-25T01:23:30.725659+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:23:30.725659+0000","last_undegraded":"2024-04-25T01:23:30.725659+0000","last_fullsized":"2024-04-25T01:23:30.725659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'9521","reported_seq":9782,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.480947+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:27:23.480947+0000","last_peered":"2024-04-25T01:27:23.480947+0000","last_clean":"2024-04-25T01:27:23.480947+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:27:23.480947+0000","last_undegraded":"2024-04-25T01:27:23.480947+0000","last_fullsized":"2024-04-25T01:27:23.480947+0000","mapping_epoch":23,"log_start":"27'1566","ondisk_log_start":"27'1566","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7955,"log_dups_size":0,"ondisk_log_size":7955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1722100,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":58674,"num_write":6884,"num_write_kb":914721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2510","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.556555+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:27:22.556555+0000","last_peered":"2024-04-25T01:27:22.556555+0000","last_clean":"2024-04-25T01:27:22.556555+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:27:22.556555+0000","last_undegraded":"2024-04-25T01:27:22.556555+0000","last_fullsized":"2024-04-25T01:27:22.556555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7379","reported_seq":7416,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:20.151024+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:26:20.151024+0000","last_peered":"2024-04-25T01:26:20.151024+0000","last_clean":"2024-04-25T01:26:20.151024+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:26:20.151024+0000","last_undegraded":"2024-04-25T01:26:20.151024+0000","last_fullsized":"2024-04-25T01:26:20.151024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7379,"log_dups_size":0,"ondisk_log_size":7379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7384,"num_write_kb":58188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9262","reported_seq":9766,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.128091+0000","last_change":"2024-04-25T01:02:13.830001+0000","last_active":"2024-04-25T01:27:23.128091+0000","last_peered":"2024-04-25T01:27:23.128091+0000","last_clean":"2024-04-25T01:27:23.128091+0000","last_became_active":"2024-04-25T01:02:13.829268+0000","last_became_peered":"2024-04-25T01:02:13.829268+0000","last_unstale":"2024-04-25T01:27:23.128091+0000","last_undegraded":"2024-04-25T01:27:23.128091+0000","last_fullsized":"2024-04-25T01:27:23.128091+0000","mapping_epoch":26,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7950,"log_dups_size":0,"ondisk_log_size":7950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:35.237193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1912054,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":468,"num_read_kb":118872,"num_write":6583,"num_write_kb":924937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2548","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.300867+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:27:28.300867+0000","last_peered":"2024-04-25T01:27:28.300867+0000","last_clean":"2024-04-25T01:27:28.300867+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:27:28.300867+0000","last_undegraded":"2024-04-25T01:27:28.300867+0000","last_fullsized":"2024-04-25T01:27:28.300867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7034","reported_seq":7071,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.339472+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:27:26.339472+0000","last_peered":"2024-04-25T01:27:26.339472+0000","last_clean":"2024-04-25T01:27:26.339472+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:27:26.339472+0000","last_undegraded":"2024-04-25T01:27:26.339472+0000","last_fullsized":"2024-04-25T01:27:26.339472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7035,"num_write_kb":58075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9439","reported_seq":10132,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.394196+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:27:23.394196+0000","last_peered":"2024-04-25T01:27:23.394196+0000","last_clean":"2024-04-25T01:27:23.394196+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:27:23.394196+0000","last_undegraded":"2024-04-25T01:27:23.394196+0000","last_fullsized":"2024-04-25T01:27:23.394196+0000","mapping_epoch":23,"log_start":"27'2038","ondisk_log_start":"27'2038","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7401,"log_dups_size":0,"ondisk_log_size":7401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3448832,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":664,"num_read_kb":168656,"num_write":6687,"num_write_kb":894893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2407","reported_seq":2442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.289927+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:27:23.289927+0000","last_peered":"2024-04-25T01:27:23.289927+0000","last_clean":"2024-04-25T01:27:23.289927+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:27:23.289927+0000","last_undegraded":"2024-04-25T01:27:23.289927+0000","last_fullsized":"2024-04-25T01:27:23.289927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2407,"log_dups_size":0,"ondisk_log_size":2407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8734","reported_seq":8771,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:37.183850+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:26:37.183850+0000","last_peered":"2024-04-25T01:26:37.183850+0000","last_clean":"2024-04-25T01:26:37.183850+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:26:37.183850+0000","last_undegraded":"2024-04-25T01:26:37.183850+0000","last_fullsized":"2024-04-25T01:26:37.183850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8734,"log_dups_size":0,"ondisk_log_size":8734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8742,"num_write_kb":79319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9292","reported_seq":10356,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.817906+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:27:27.817906+0000","last_peered":"2024-04-25T01:27:27.817906+0000","last_clean":"2024-04-25T01:27:27.817906+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:27:27.817906+0000","last_undegraded":"2024-04-25T01:27:27.817906+0000","last_fullsized":"2024-04-25T01:27:27.817906+0000","mapping_epoch":23,"log_start":"27'1886","ondisk_log_start":"27'1886","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7406,"log_dups_size":0,"ondisk_log_size":7406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187142,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1035,"num_read_kb":262890,"num_write":6554,"num_write_kb":1048321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.190123+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:27:23.190123+0000","last_peered":"2024-04-25T01:27:23.190123+0000","last_clean":"2024-04-25T01:27:23.190123+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:27:23.190123+0000","last_undegraded":"2024-04-25T01:27:23.190123+0000","last_fullsized":"2024-04-25T01:27:23.190123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5708","reported_seq":5745,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.330594+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:27:26.330594+0000","last_peered":"2024-04-25T01:27:26.330594+0000","last_clean":"2024-04-25T01:27:26.330594+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:27:26.330594+0000","last_undegraded":"2024-04-25T01:27:26.330594+0000","last_fullsized":"2024-04-25T01:27:26.330594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":44693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'8855","reported_seq":9273,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.523452+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:27:27.523452+0000","last_peered":"2024-04-25T01:27:27.523452+0000","last_clean":"2024-04-25T01:27:27.523452+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:27:27.523452+0000","last_undegraded":"2024-04-25T01:27:27.523452+0000","last_fullsized":"2024-04-25T01:27:27.523452+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7245,"log_dups_size":0,"ondisk_log_size":7245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":412999,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":394,"num_read_kb":100076,"num_write":6145,"num_write_kb":854299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2584","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.380863+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:27:23.380863+0000","last_peered":"2024-04-25T01:27:23.380863+0000","last_clean":"2024-04-25T01:27:23.380863+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:27:23.380863+0000","last_undegraded":"2024-04-25T01:27:23.380863+0000","last_fullsized":"2024-04-25T01:27:23.380863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2584,"log_dups_size":0,"ondisk_log_size":2584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'5129","reported_seq":5166,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:15.200879+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:27:15.200879+0000","last_peered":"2024-04-25T01:27:15.200879+0000","last_clean":"2024-04-25T01:27:15.200879+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:27:15.200879+0000","last_undegraded":"2024-04-25T01:27:15.200879+0000","last_fullsized":"2024-04-25T01:27:15.200879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5129,"log_dups_size":0,"ondisk_log_size":5129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":40185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9360","reported_seq":9696,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.479124+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:27:23.479124+0000","last_peered":"2024-04-25T01:27:23.479124+0000","last_clean":"2024-04-25T01:27:23.479124+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:27:23.479124+0000","last_undegraded":"2024-04-25T01:27:23.479124+0000","last_fullsized":"2024-04-25T01:27:23.479124+0000","mapping_epoch":23,"log_start":"27'2023","ondisk_log_start":"27'2023","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6280246,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":308,"num_read_kb":78232,"num_write":6676,"num_write_kb":1180887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2450","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.397998+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:27:23.397998+0000","last_peered":"2024-04-25T01:27:23.397998+0000","last_clean":"2024-04-25T01:27:23.397998+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:27:23.397998+0000","last_undegraded":"2024-04-25T01:27:23.397998+0000","last_fullsized":"2024-04-25T01:27:23.397998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4840","reported_seq":4877,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:14.595711+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:27:14.595711+0000","last_peered":"2024-04-25T01:27:14.595711+0000","last_clean":"2024-04-25T01:27:14.595711+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:27:14.595711+0000","last_undegraded":"2024-04-25T01:27:14.595711+0000","last_fullsized":"2024-04-25T01:27:14.595711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4844,"num_write_kb":44079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9089","reported_seq":9590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.255740+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:27:23.255740+0000","last_peered":"2024-04-25T01:27:23.255740+0000","last_clean":"2024-04-25T01:27:23.255740+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:27:23.255740+0000","last_undegraded":"2024-04-25T01:27:23.255740+0000","last_fullsized":"2024-04-25T01:27:23.255740+0000","mapping_epoch":23,"log_start":"27'2828","ondisk_log_start":"27'2828","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2716911,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":473,"num_read_kb":120142,"num_write":6358,"num_write_kb":909053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2554","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.084782+0000","last_change":"2024-04-25T01:02:13.744191+0000","last_active":"2024-04-25T01:27:23.084782+0000","last_peered":"2024-04-25T01:27:23.084782+0000","last_clean":"2024-04-25T01:27:23.084782+0000","last_became_active":"2024-04-25T01:02:13.743832+0000","last_became_peered":"2024-04-25T01:02:13.743832+0000","last_unstale":"2024-04-25T01:27:23.084782+0000","last_undegraded":"2024-04-25T01:27:23.084782+0000","last_fullsized":"2024-04-25T01:27:23.084782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"27'2887","reported_seq":2924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.345321+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:27:26.345321+0000","last_peered":"2024-04-25T01:27:26.345321+0000","last_clean":"2024-04-25T01:27:26.345321+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:27:26.345321+0000","last_undegraded":"2024-04-25T01:27:26.345321+0000","last_fullsized":"2024-04-25T01:27:26.345321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2887,"log_dups_size":0,"ondisk_log_size":2887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2884,"num_write_kb":30449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9218","reported_seq":9460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.036523+0000","last_change":"2024-04-25T01:02:13.830353+0000","last_active":"2024-04-25T01:27:23.036523+0000","last_peered":"2024-04-25T01:27:23.036523+0000","last_clean":"2024-04-25T01:27:23.036523+0000","last_became_active":"2024-04-25T01:02:13.829847+0000","last_became_peered":"2024-04-25T01:02:13.829847+0000","last_unstale":"2024-04-25T01:27:23.036523+0000","last_undegraded":"2024-04-25T01:27:23.036523+0000","last_fullsized":"2024-04-25T01:27:23.036523+0000","mapping_epoch":26,"log_start":"27'2967","ondisk_log_start":"27'2967","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:00.933599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1903266,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":52578,"num_write":6608,"num_write_kb":922819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2517","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.153861+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:27:23.153861+0000","last_peered":"2024-04-25T01:27:23.153861+0000","last_clean":"2024-04-25T01:27:23.153861+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:27:23.153861+0000","last_undegraded":"2024-04-25T01:27:23.153861+0000","last_fullsized":"2024-04-25T01:27:23.153861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2517,"log_dups_size":0,"ondisk_log_size":2517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6766","reported_seq":6803,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:03.691415+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:27:03.691415+0000","last_peered":"2024-04-25T01:27:03.691415+0000","last_clean":"2024-04-25T01:27:03.691415+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:27:03.691415+0000","last_undegraded":"2024-04-25T01:27:03.691415+0000","last_fullsized":"2024-04-25T01:27:03.691415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6766,"log_dups_size":0,"ondisk_log_size":6766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6774,"num_write_kb":57696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9611","reported_seq":9984,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.485448+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:27:28.485448+0000","last_peered":"2024-04-25T01:27:28.485448+0000","last_clean":"2024-04-25T01:27:28.485448+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:27:28.485448+0000","last_undegraded":"2024-04-25T01:27:28.485448+0000","last_fullsized":"2024-04-25T01:27:28.485448+0000","mapping_epoch":23,"log_start":"27'2214","ondisk_log_start":"27'2214","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2536524,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":349,"num_read_kb":88646,"num_write":6979,"num_write_kb":1079633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2454","reported_seq":2488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.321625+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:27:23.321625+0000","last_peered":"2024-04-25T01:27:23.321625+0000","last_clean":"2024-04-25T01:27:23.321625+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:27:23.321625+0000","last_undegraded":"2024-04-25T01:27:23.321625+0000","last_fullsized":"2024-04-25T01:27:23.321625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2454,"log_dups_size":0,"ondisk_log_size":2454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10280","reported_seq":10909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:15.207946+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:27:15.207946+0000","last_peered":"2024-04-25T01:27:15.207946+0000","last_clean":"2024-04-25T01:27:15.207946+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:27:15.207946+0000","last_undegraded":"2024-04-25T01:27:15.207946+0000","last_fullsized":"2024-04-25T01:27:15.207946+0000","mapping_epoch":18,"log_start":"27'2300","ondisk_log_start":"27'2300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":594,"num_read_kb":4482,"num_write":10274,"num_write_kb":76426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9142","reported_seq":9483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.928290+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:27:27.928290+0000","last_peered":"2024-04-25T01:27:27.928290+0000","last_clean":"2024-04-25T01:27:27.928290+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:27:27.928290+0000","last_undegraded":"2024-04-25T01:27:27.928290+0000","last_fullsized":"2024-04-25T01:27:27.928290+0000","mapping_epoch":23,"log_start":"27'1734","ondisk_log_start":"27'1734","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7408,"log_dups_size":0,"ondisk_log_size":7408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1592989,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":312,"num_read_kb":79248,"num_write":6479,"num_write_kb":938399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2545","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.035246+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:27:23.035246+0000","last_peered":"2024-04-25T01:27:23.035246+0000","last_clean":"2024-04-25T01:27:23.035246+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:27:23.035246+0000","last_undegraded":"2024-04-25T01:27:23.035246+0000","last_fullsized":"2024-04-25T01:27:23.035246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2545,"log_dups_size":0,"ondisk_log_size":2545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6456","reported_seq":6492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.396435+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:27:26.396435+0000","last_peered":"2024-04-25T01:27:26.396435+0000","last_clean":"2024-04-25T01:27:26.396435+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:27:26.396435+0000","last_undegraded":"2024-04-25T01:27:26.396435+0000","last_fullsized":"2024-04-25T01:27:26.396435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6456,"log_dups_size":0,"ondisk_log_size":6456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6471,"num_write_kb":41578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'8950","reported_seq":9484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.661822+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:27:27.661822+0000","last_peered":"2024-04-25T01:27:27.661822+0000","last_clean":"2024-04-25T01:27:27.661822+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:27:27.661822+0000","last_undegraded":"2024-04-25T01:27:27.661822+0000","last_fullsized":"2024-04-25T01:27:27.661822+0000","mapping_epoch":23,"log_start":"27'1803","ondisk_log_start":"27'1803","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7147,"log_dups_size":0,"ondisk_log_size":7147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2560955,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":508,"num_read_kb":129032,"num_write":6252,"num_write_kb":1000416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2560","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.148657+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:27:23.148657+0000","last_peered":"2024-04-25T01:27:23.148657+0000","last_clean":"2024-04-25T01:27:23.148657+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:27:23.148657+0000","last_undegraded":"2024-04-25T01:27:23.148657+0000","last_fullsized":"2024-04-25T01:27:23.148657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2560,"log_dups_size":0,"ondisk_log_size":2560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6614","reported_seq":6650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.363784+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:27:26.363784+0000","last_peered":"2024-04-25T01:27:26.363784+0000","last_clean":"2024-04-25T01:27:26.363784+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:27:26.363784+0000","last_undegraded":"2024-04-25T01:27:26.363784+0000","last_fullsized":"2024-04-25T01:27:26.363784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6614,"log_dups_size":0,"ondisk_log_size":6614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6608,"num_write_kb":53556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9193","reported_seq":9400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.462589+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:27:23.462589+0000","last_peered":"2024-04-25T01:27:23.462589+0000","last_clean":"2024-04-25T01:27:23.462589+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:27:23.462589+0000","last_undegraded":"2024-04-25T01:27:23.462589+0000","last_fullsized":"2024-04-25T01:27:23.462589+0000","mapping_epoch":23,"log_start":"27'2910","ondisk_log_start":"27'2910","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1149248,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6520,"num_write_kb":962512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2543","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.312816+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:27:28.312816+0000","last_peered":"2024-04-25T01:27:28.312816+0000","last_clean":"2024-04-25T01:27:28.312816+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:27:28.312816+0000","last_undegraded":"2024-04-25T01:27:28.312816+0000","last_fullsized":"2024-04-25T01:27:28.312816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2543,"log_dups_size":0,"ondisk_log_size":2543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6620","reported_seq":7475,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:25:50.038282+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:25:50.038282+0000","last_peered":"2024-04-25T01:25:50.038282+0000","last_clean":"2024-04-25T01:25:50.038282+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:25:50.038282+0000","last_undegraded":"2024-04-25T01:25:50.038282+0000","last_fullsized":"2024-04-25T01:25:50.038282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":821,"num_read_kb":4618,"num_write":6622,"num_write_kb":49469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9979","reported_seq":10043,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.444337+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:27:23.444337+0000","last_peered":"2024-04-25T01:27:23.444337+0000","last_clean":"2024-04-25T01:27:23.444337+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:27:23.444337+0000","last_undegraded":"2024-04-25T01:27:23.444337+0000","last_fullsized":"2024-04-25T01:27:23.444337+0000","mapping_epoch":23,"log_start":"27'3717","ondisk_log_start":"27'3717","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1707840,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":35,"num_read_kb":8890,"num_write":7259,"num_write_kb":1063948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9051","reported_seq":9308,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.341237+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:27:23.341237+0000","last_peered":"2024-04-25T01:27:23.341237+0000","last_clean":"2024-04-25T01:27:23.341237+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:27:23.341237+0000","last_undegraded":"2024-04-25T01:27:23.341237+0000","last_fullsized":"2024-04-25T01:27:23.341237+0000","mapping_epoch":23,"log_start":"27'1719","ondisk_log_start":"27'1719","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":227,"num_read_kb":57658,"num_write":6452,"num_write_kb":909285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2475","reported_seq":2509,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.998206+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:27:22.998206+0000","last_peered":"2024-04-25T01:27:22.998206+0000","last_clean":"2024-04-25T01:27:22.998206+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:27:22.998206+0000","last_undegraded":"2024-04-25T01:27:22.998206+0000","last_fullsized":"2024-04-25T01:27:22.998206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9899","reported_seq":9935,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322490+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:27:26.322490+0000","last_peered":"2024-04-25T01:27:26.322490+0000","last_clean":"2024-04-25T01:27:26.322490+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:27:26.322490+0000","last_undegraded":"2024-04-25T01:27:26.322490+0000","last_fullsized":"2024-04-25T01:27:26.322490+0000","mapping_epoch":18,"log_start":"27'2500","ondisk_log_start":"27'2500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":68120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8927","reported_seq":9786,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.374438+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:27:23.374438+0000","last_peered":"2024-04-25T01:27:23.374438+0000","last_clean":"2024-04-25T01:27:23.374438+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:27:23.374438+0000","last_undegraded":"2024-04-25T01:27:23.374438+0000","last_fullsized":"2024-04-25T01:27:23.374438+0000","mapping_epoch":23,"log_start":"27'1000","ondisk_log_start":"27'1000","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7927,"log_dups_size":0,"ondisk_log_size":7927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2619556,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":829,"num_read_kb":210566,"num_write":6293,"num_write_kb":892044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2577","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.071456+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:27:23.071456+0000","last_peered":"2024-04-25T01:27:23.071456+0000","last_clean":"2024-04-25T01:27:23.071456+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:27:23.071456+0000","last_undegraded":"2024-04-25T01:27:23.071456+0000","last_fullsized":"2024-04-25T01:27:23.071456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5341","reported_seq":5377,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:13.925537+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:26:13.925537+0000","last_peered":"2024-04-25T01:26:13.925537+0000","last_clean":"2024-04-25T01:26:13.925537+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:26:13.925537+0000","last_undegraded":"2024-04-25T01:26:13.925537+0000","last_fullsized":"2024-04-25T01:26:13.925537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5341,"num_write_kb":44512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9496","reported_seq":9532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:14.123973+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:26:14.123973+0000","last_peered":"2024-04-25T01:26:14.123973+0000","last_clean":"2024-04-25T01:26:14.123973+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:26:14.123973+0000","last_undegraded":"2024-04-25T01:26:14.123973+0000","last_fullsized":"2024-04-25T01:26:14.123973+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":9507,"num_write_kb":71968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2515","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.271703+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:27:23.271703+0000","last_peered":"2024-04-25T01:27:23.271703+0000","last_clean":"2024-04-25T01:27:23.271703+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:27:23.271703+0000","last_undegraded":"2024-04-25T01:27:23.271703+0000","last_fullsized":"2024-04-25T01:27:23.271703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9444","reported_seq":9599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.419393+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:27:23.419393+0000","last_peered":"2024-04-25T01:27:23.419393+0000","last_clean":"2024-04-25T01:27:23.419393+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:27:23.419393+0000","last_undegraded":"2024-04-25T01:27:23.419393+0000","last_fullsized":"2024-04-25T01:27:23.419393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4296704,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":127,"num_read_kb":32258,"num_write":6733,"num_write_kb":1000281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8023","reported_seq":8059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:24:51.994967+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:24:51.994967+0000","last_peered":"2024-04-25T01:24:51.994967+0000","last_clean":"2024-04-25T01:24:51.994967+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:24:51.994967+0000","last_undegraded":"2024-04-25T01:24:51.994967+0000","last_fullsized":"2024-04-25T01:24:51.994967+0000","mapping_epoch":18,"log_start":"27'1700","ondisk_log_start":"27'1700","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8018,"num_write_kb":66584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2558","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.048774+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:27:23.048774+0000","last_peered":"2024-04-25T01:27:23.048774+0000","last_clean":"2024-04-25T01:27:23.048774+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:27:23.048774+0000","last_undegraded":"2024-04-25T01:27:23.048774+0000","last_fullsized":"2024-04-25T01:27:23.048774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9429","reported_seq":9825,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.252529+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:27:23.252529+0000","last_peered":"2024-04-25T01:27:23.252529+0000","last_clean":"2024-04-25T01:27:23.252529+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:27:23.252529+0000","last_undegraded":"2024-04-25T01:27:23.252529+0000","last_fullsized":"2024-04-25T01:27:23.252529+0000","mapping_epoch":23,"log_start":"27'3125","ondisk_log_start":"27'3125","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6304,"log_dups_size":0,"ondisk_log_size":6304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6485238,"num_objects":15,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":368,"num_read_kb":93472,"num_write":6715,"num_write_kb":1049905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2448","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.142813+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:27:23.142813+0000","last_peered":"2024-04-25T01:27:23.142813+0000","last_clean":"2024-04-25T01:27:23.142813+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:27:23.142813+0000","last_undegraded":"2024-04-25T01:27:23.142813+0000","last_fullsized":"2024-04-25T01:27:23.142813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2448,"log_dups_size":0,"ondisk_log_size":2448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2578","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.052048+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:27:23.052048+0000","last_peered":"2024-04-25T01:27:23.052048+0000","last_clean":"2024-04-25T01:27:23.052048+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:27:23.052048+0000","last_undegraded":"2024-04-25T01:27:23.052048+0000","last_fullsized":"2024-04-25T01:27:23.052048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2493","reported_seq":2529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.401417+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:27:28.401417+0000","last_peered":"2024-04-25T01:27:28.401417+0000","last_clean":"2024-04-25T01:27:28.401417+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:27:28.401417+0000","last_undegraded":"2024-04-25T01:27:28.401417+0000","last_fullsized":"2024-04-25T01:27:28.401417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2562","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.122836+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:27:23.122836+0000","last_peered":"2024-04-25T01:27:23.122836+0000","last_clean":"2024-04-25T01:27:23.122836+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:27:23.122836+0000","last_undegraded":"2024-04-25T01:27:23.122836+0000","last_fullsized":"2024-04-25T01:27:23.122836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.340997+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:27:23.340997+0000","last_peered":"2024-04-25T01:27:23.340997+0000","last_clean":"2024-04-25T01:27:23.340997+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:27:23.340997+0000","last_undegraded":"2024-04-25T01:27:23.340997+0000","last_fullsized":"2024-04-25T01:27:23.340997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2445","reported_seq":2480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.369622+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:27:23.369622+0000","last_peered":"2024-04-25T01:27:23.369622+0000","last_clean":"2024-04-25T01:27:23.369622+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:27:23.369622+0000","last_undegraded":"2024-04-25T01:27:23.369622+0000","last_fullsized":"2024-04-25T01:27:23.369622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2573","reported_seq":2608,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.402625+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:27:23.402625+0000","last_peered":"2024-04-25T01:27:23.402625+0000","last_clean":"2024-04-25T01:27:23.402625+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:27:23.402625+0000","last_undegraded":"2024-04-25T01:27:23.402625+0000","last_fullsized":"2024-04-25T01:27:23.402625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2538","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.170149+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:27:23.170149+0000","last_peered":"2024-04-25T01:27:23.170149+0000","last_clean":"2024-04-25T01:27:23.170149+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:27:23.170149+0000","last_undegraded":"2024-04-25T01:27:23.170149+0000","last_fullsized":"2024-04-25T01:27:23.170149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2538,"log_dups_size":0,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2529","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.289774+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:27:28.289774+0000","last_peered":"2024-04-25T01:27:28.289774+0000","last_clean":"2024-04-25T01:27:28.289774+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:27:28.289774+0000","last_undegraded":"2024-04-25T01:27:28.289774+0000","last_fullsized":"2024-04-25T01:27:28.289774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2500","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.365002+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:27:23.365002+0000","last_peered":"2024-04-25T01:27:23.365002+0000","last_clean":"2024-04-25T01:27:23.365002+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:27:23.365002+0000","last_undegraded":"2024-04-25T01:27:23.365002+0000","last_fullsized":"2024-04-25T01:27:23.365002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2557","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.475740+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:27:28.475740+0000","last_peered":"2024-04-25T01:27:28.475740+0000","last_clean":"2024-04-25T01:27:28.475740+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:27:28.475740+0000","last_undegraded":"2024-04-25T01:27:28.475740+0000","last_fullsized":"2024-04-25T01:27:28.475740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2567","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.273548+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:27:23.273548+0000","last_peered":"2024-04-25T01:27:23.273548+0000","last_clean":"2024-04-25T01:27:23.273548+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:27:23.273548+0000","last_undegraded":"2024-04-25T01:27:23.273548+0000","last_fullsized":"2024-04-25T01:27:23.273548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2572","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.152624+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:27:23.152624+0000","last_peered":"2024-04-25T01:27:23.152624+0000","last_clean":"2024-04-25T01:27:23.152624+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:27:23.152624+0000","last_undegraded":"2024-04-25T01:27:23.152624+0000","last_fullsized":"2024-04-25T01:27:23.152624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2572,"log_dups_size":0,"ondisk_log_size":2572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2573","reported_seq":2608,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.393729+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:27:23.393729+0000","last_peered":"2024-04-25T01:27:23.393729+0000","last_clean":"2024-04-25T01:27:23.393729+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:27:23.393729+0000","last_undegraded":"2024-04-25T01:27:23.393729+0000","last_fullsized":"2024-04-25T01:27:23.393729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2512","reported_seq":2547,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.365445+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:27:23.365445+0000","last_peered":"2024-04-25T01:27:23.365445+0000","last_clean":"2024-04-25T01:27:23.365445+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:27:23.365445+0000","last_undegraded":"2024-04-25T01:27:23.365445+0000","last_fullsized":"2024-04-25T01:27:23.365445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2512,"log_dups_size":0,"ondisk_log_size":2512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2588","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.349289+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:27:28.349289+0000","last_peered":"2024-04-25T01:27:28.349289+0000","last_clean":"2024-04-25T01:27:28.349289+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:27:28.349289+0000","last_undegraded":"2024-04-25T01:27:28.349289+0000","last_fullsized":"2024-04-25T01:27:28.349289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2509","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.305277+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:27:28.305277+0000","last_peered":"2024-04-25T01:27:28.305277+0000","last_clean":"2024-04-25T01:27:28.305277+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:27:28.305277+0000","last_undegraded":"2024-04-25T01:27:28.305277+0000","last_fullsized":"2024-04-25T01:27:28.305277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.045603+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:27:23.045603+0000","last_peered":"2024-04-25T01:27:23.045603+0000","last_clean":"2024-04-25T01:27:23.045603+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:27:23.045603+0000","last_undegraded":"2024-04-25T01:27:23.045603+0000","last_fullsized":"2024-04-25T01:27:23.045603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2482","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.837016+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:27:22.837016+0000","last_peered":"2024-04-25T01:27:22.837016+0000","last_clean":"2024-04-25T01:27:22.837016+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:27:22.837016+0000","last_undegraded":"2024-04-25T01:27:22.837016+0000","last_fullsized":"2024-04-25T01:27:22.837016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2471","reported_seq":2505,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.182821+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:27:23.182821+0000","last_peered":"2024-04-25T01:27:23.182821+0000","last_clean":"2024-04-25T01:27:23.182821+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:27:23.182821+0000","last_undegraded":"2024-04-25T01:27:23.182821+0000","last_fullsized":"2024-04-25T01:27:23.182821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2431","reported_seq":2465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.795020+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:27:22.795020+0000","last_peered":"2024-04-25T01:27:22.795020+0000","last_clean":"2024-04-25T01:27:22.795020+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:27:22.795020+0000","last_undegraded":"2024-04-25T01:27:22.795020+0000","last_fullsized":"2024-04-25T01:27:22.795020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2431,"log_dups_size":0,"ondisk_log_size":2431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2464","reported_seq":2503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.288941+0000","last_change":"2024-04-25T01:02:13.749183+0000","last_active":"2024-04-25T01:27:23.288941+0000","last_peered":"2024-04-25T01:27:23.288941+0000","last_clean":"2024-04-25T01:27:23.288941+0000","last_became_active":"2024-04-25T01:02:13.748901+0000","last_became_peered":"2024-04-25T01:02:13.748901+0000","last_unstale":"2024-04-25T01:27:23.288941+0000","last_undegraded":"2024-04-25T01:27:23.288941+0000","last_fullsized":"2024-04-25T01:27:23.288941+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2464,"log_dups_size":0,"ondisk_log_size":2464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:54.284115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2596","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.443677+0000","last_change":"2024-04-25T01:02:13.750135+0000","last_active":"2024-04-25T01:27:23.443677+0000","last_peered":"2024-04-25T01:27:23.443677+0000","last_clean":"2024-04-25T01:27:23.443677+0000","last_became_active":"2024-04-25T01:02:13.749859+0000","last_became_peered":"2024-04-25T01:02:13.749859+0000","last_unstale":"2024-04-25T01:27:23.443677+0000","last_undegraded":"2024-04-25T01:27:23.443677+0000","last_fullsized":"2024-04-25T01:27:23.443677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:18.550102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2552","reported_seq":2586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.351165+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:27:23.351165+0000","last_peered":"2024-04-25T01:27:23.351165+0000","last_clean":"2024-04-25T01:27:23.351165+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:27:23.351165+0000","last_undegraded":"2024-04-25T01:27:23.351165+0000","last_fullsized":"2024-04-25T01:27:23.351165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2552,"log_dups_size":0,"ondisk_log_size":2552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2498","reported_seq":2518,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.384142+0000","last_change":"2024-04-25T01:02:13.828931+0000","last_active":"2024-04-25T01:27:23.384142+0000","last_peered":"2024-04-25T01:27:23.384142+0000","last_clean":"2024-04-25T01:27:23.384142+0000","last_became_active":"2024-04-25T01:02:13.828599+0000","last_became_peered":"2024-04-25T01:02:13.828599+0000","last_unstale":"2024-04-25T01:27:23.384142+0000","last_undegraded":"2024-04-25T01:27:23.384142+0000","last_fullsized":"2024-04-25T01:27:23.384142+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:20.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.39","version":"27'2514","reported_seq":2548,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.084091+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:27:23.084091+0000","last_peered":"2024-04-25T01:27:23.084091+0000","last_clean":"2024-04-25T01:27:23.084091+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:27:23.084091+0000","last_undegraded":"2024-04-25T01:27:23.084091+0000","last_fullsized":"2024-04-25T01:27:23.084091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2468","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.359740+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:27:28.359740+0000","last_peered":"2024-04-25T01:27:28.359740+0000","last_clean":"2024-04-25T01:27:28.359740+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:27:28.359740+0000","last_undegraded":"2024-04-25T01:27:28.359740+0000","last_fullsized":"2024-04-25T01:27:28.359740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2468,"log_dups_size":0,"ondisk_log_size":2468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2492","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.629304+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:27:22.629304+0000","last_peered":"2024-04-25T01:27:22.629304+0000","last_clean":"2024-04-25T01:27:22.629304+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:27:22.629304+0000","last_undegraded":"2024-04-25T01:27:22.629304+0000","last_fullsized":"2024-04-25T01:27:22.629304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2492,"log_dups_size":0,"ondisk_log_size":2492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2557","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.305843+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:27:28.305843+0000","last_peered":"2024-04-25T01:27:28.305843+0000","last_clean":"2024-04-25T01:27:28.305843+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:27:28.305843+0000","last_undegraded":"2024-04-25T01:27:28.305843+0000","last_fullsized":"2024-04-25T01:27:28.305843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2606","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.217124+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:27:23.217124+0000","last_peered":"2024-04-25T01:27:23.217124+0000","last_clean":"2024-04-25T01:27:23.217124+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:27:23.217124+0000","last_undegraded":"2024-04-25T01:27:23.217124+0000","last_fullsized":"2024-04-25T01:27:23.217124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2458","reported_seq":2494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.190500+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:27:23.190500+0000","last_peered":"2024-04-25T01:27:23.190500+0000","last_clean":"2024-04-25T01:27:23.190500+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:27:23.190500+0000","last_undegraded":"2024-04-25T01:27:23.190500+0000","last_fullsized":"2024-04-25T01:27:23.190500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2434","reported_seq":2470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.399721+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:27:23.399721+0000","last_peered":"2024-04-25T01:27:23.399721+0000","last_clean":"2024-04-25T01:27:23.399721+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:27:23.399721+0000","last_undegraded":"2024-04-25T01:27:23.399721+0000","last_fullsized":"2024-04-25T01:27:23.399721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":78633850,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":12020,"num_read_kb":3053080,"num_write":211822,"num_write_kb":30669379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160497664,"data_stored":160497664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232144,"ondisk_log_size":232144,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161709,"ondisk_log_size":161709,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":230335553,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":3110,"num_read_kb":19066,"num_write":209236,"num_write_kb":1676068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":70,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459538432,"data_stored":459478144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194116,"ondisk_log_size":194116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575165,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":956092,"kb_used_data":58612,"kb_used_omap":1,"kb_used_meta":897470,"kb_avail":93415748,"statfs":{"total":96636764160,"available":95657725952,"internally_reserved":0,"allocated":60018688,"data_stored":59837569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":919009738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1202892,"kb_used_data":89156,"kb_used_omap":1,"kb_used_meta":1113726,"kb_avail":93168948,"statfs":{"total":96636764160,"available":95405002752,"internally_reserved":0,"allocated":91295744,"data_stored":91116082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1140455878},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1009360,"kb_used_data":57864,"kb_used_omap":1,"kb_used_meta":951486,"kb_avail":93362480,"statfs":{"total":96636764160,"available":95603179520,"internally_reserved":0,"allocated":59252736,"data_stored":59074738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":974322117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575162,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1013728,"kb_used_data":80280,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93358112,"statfs":{"total":96636764160,"available":95598706688,"internally_reserved":0,"allocated":82206720,"data_stored":82022774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":955840965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1154908,"kb_used_data":99284,"kb_used_omap":1,"kb_used_meta":1055614,"kb_avail":93216932,"statfs":{"total":96636764160,"available":95454138368,"internally_reserved":0,"allocated":101666816,"data_stored":101487512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1080949191},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694676,"kb_used_data":72140,"kb_used_omap":1,"kb_used_meta":622526,"kb_avail":93677164,"statfs":{"total":96636764160,"available":95925415936,"internally_reserved":0,"allocated":73871360,"data_stored":73699576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637467077},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1251780,"kb_used_data":79612,"kb_used_omap":1,"kb_used_meta":1172158,"kb_avail":93120060,"statfs":{"total":96636764160,"available":95354941440,"internally_reserved":0,"allocated":81522688,"data_stored":81341823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1200290245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850464,"kb_used_data":71128,"kb_used_omap":1,"kb_used_meta":779326,"kb_avail":93521376,"statfs":{"total":96636764160,"available":95765889024,"internally_reserved":0,"allocated":72835072,"data_stored":72648086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":798030277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54538240,"data_stored":54526590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60657664,"data_stored":60648583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54525952,"data_stored":54525952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40861696,"data_stored":40855482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":67837952,"data_stored":67830074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":46878720,"data_stored":46872937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17412096,"data_stored":17412096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26370048,"data_stored":26370048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19054592,"data_stored":19054592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27361280,"data_stored":27361280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17469440,"data_stored":17469440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":17408000,"data_stored":17408000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23166976,"data_stored":23166976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12255232,"data_stored":12255232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:27:29.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:27:29.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.358+0000 7f341f59e640 1 Processor -- start 2024-04-25T01:27:29.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.358+0000 7f341f59e640 1 -- start start 2024-04-25T01:27:29.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f341f59e640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f34200accd0 0x7f3420159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f341f59e640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3420070d50 con 0x7f34200c2b10 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f341f59e640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3420071030 con 0x7f342006cc30 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f341f59e640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f3420071680 con 0x7f34200accd0 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f341ed9d640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f342006cc30 0x7f34201567e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:44226/0 (socket says 172.21.15.129:44226) 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f341ed9d640 1 -- 172.21.15.129:0/2226771302 learned_addr learned my addr 172.21.15.129:0/2226771302 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.359+0000 7f3417fff640 1 --2- 172.21.15.129:0/2226771302 >> v2:172.21.15.171:3300/0 conn(0x7f34200accd0 0x7f3420159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.360+0000 7f3417fff640 1 -- 172.21.15.129:0/2226771302 >> v1:172.21.15.171:6789/0 conn(0x7f342006cc30 legacy=0x7f34201567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.360+0000 7f3417fff640 1 -- 172.21.15.129:0/2226771302 >> v1:172.21.15.129:6789/0 conn(0x7f34200c2b10 legacy=0x7f3420071b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.360+0000 7f3417fff640 1 -- 172.21.15.129:0/2226771302 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3420071310 con 0x7f34200accd0 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.361+0000 7f3417fff640 1 --2- 172.21.15.129:0/2226771302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34200accd0 0x7f3420159fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f34080097a0 tx=0x7f340802a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6cb115bc097e149 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.361+0000 7f341dd9b640 1 -- 172.21.15.129:0/2226771302 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408037070 con 0x7f34200accd0 2024-04-25T01:27:29.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.361+0000 7f341dd9b640 1 -- 172.21.15.129:0/2226771302 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408032440 con 0x7f34200accd0 2024-04-25T01:27:29.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.362+0000 7f341f59e640 1 -- 172.21.15.129:0/2226771302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34200accd0 msgr2=0x7f3420159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.362+0000 7f341f59e640 1 --2- 172.21.15.129:0/2226771302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34200accd0 0x7f3420159fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f34080097a0 tx=0x7f340802a160 comp rx=0 tx=0).stop 2024-04-25T01:27:29.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.362+0000 7f341f59e640 1 -- 172.21.15.129:0/2226771302 shutdown_connections 2024-04-25T01:27:29.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.362+0000 7f341f59e640 1 --2- 172.21.15.129:0/2226771302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34200accd0 0x7f3420159fb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.362+0000 7f341f59e640 1 -- 172.21.15.129:0/2226771302 >> 172.21.15.129:0/2226771302 conn(0x7f34200c3990 msgr2=0x7f34200c3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:29.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.366+0000 7f341f59e640 1 -- 172.21.15.129:0/2226771302 shutdown_connections 2024-04-25T01:27:29.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.370+0000 7f341f59e640 1 -- 172.21.15.129:0/2226771302 wait complete. 2024-04-25T01:27:29.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.372+0000 7f341f59e640 1 Processor -- start 2024-04-25T01:27:29.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.376+0000 7f341f59e640 1 -- start start 2024-04-25T01:27:29.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.381+0000 7f341f59e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f342006cc30 0x7f342014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.381+0000 7f341f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 0x7f34201f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.381+0000 7f341f59e640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34201f2bb0 0x7f34200abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.381+0000 7f341f59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34200c2ef0 con 0x7f342014e3a0 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.381+0000 7f341f59e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3420073130 con 0x7f342006cc30 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.381+0000 7f341f59e640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f34200c3070 con 0x7f34201f2bb0 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 0x7f34201f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 0x7f34201f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55328/0 (socket says 172.21.15.129:55328) 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 -- 172.21.15.129:0/1983696373 learned_addr learned my addr 172.21.15.129:0/1983696373 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:29.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f341e59c640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f342006cc30 0x7f342014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f341ed9d640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34201f2bb0 0x7f34200abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 -- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34201f2bb0 msgr2=0x7f34200abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34201f2bb0 0x7f34200abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 -- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f342006cc30 msgr2=0x7f342014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f342006cc30 0x7f342014da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.382+0000 7f3417fff640 1 -- 172.21.15.129:0/1983696373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342015bc70 con 0x7f342014e3a0 2024-04-25T01:27:29.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.383+0000 7f3417fff640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 0x7f34201f2670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f340802a750 tx=0x7f34080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.384+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408037050 con 0x7f342014e3a0 2024-04-25T01:27:29.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.384+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3408043070 con 0x7f342014e3a0 2024-04-25T01:27:29.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.384+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340803e650 con 0x7f342014e3a0 2024-04-25T01:27:29.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.384+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201f3120 con 0x7f342014e3a0 2024-04-25T01:27:29.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.384+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34200aa420 con 0x7f342014e3a0 2024-04-25T01:27:29.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.387+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3408032500 con 0x7f342014e3a0 2024-04-25T01:27:29.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.388+0000 7f3416ffd640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f340006bf00 0x7f340006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.388+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f34080ac060 con 0x7f342014e3a0 2024-04-25T01:27:29.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.390+0000 7f341e59c640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f340006bf00 0x7f340006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.391+0000 7f341e59c640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f340006bf00 0x7f340006e3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3420074a60 tx=0x7f34100090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.392+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3404005290 con 0x7f342014e3a0 2024-04-25T01:27:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.401+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3408079fa0 con 0x7f342014e3a0 2024-04-25T01:27:29.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.691+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3404000bc0 con 0x7f340006bf00 2024-04-25T01:27:29.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.699+0000 7f3416ffd640 1 -- 172.21.15.129:0/1983696373 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336161 (secure 0 0 0) 0x7f3404000bc0 con 0x7f340006bf00 2024-04-25T01:27:29.683 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:29.684 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f340006bf00 msgr2=0x7f340006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f340006bf00 0x7f340006e3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3420074a60 tx=0x7f34100090e0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 msgr2=0x7f34201f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 0x7f34201f2670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f340802a750 tx=0x7f34080046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 shutdown_connections 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f340006bf00 0x7f340006e3c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f34201f2bb0 0x7f34200abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f342014e3a0 0x7f34201f2670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 --2- 172.21.15.129:0/1983696373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f342006cc30 0x7f342014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 >> 172.21.15.129:0/1983696373 conn(0x7f34200c3990 msgr2=0x7f3420158780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 shutdown_connections 2024-04-25T01:27:29.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.702+0000 7f341f59e640 1 -- 172.21.15.129:0/1983696373 wait complete. 2024-04-25T01:27:29.693 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":805,"stamp":"2024-04-25T01:27:29.057652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":309559771,"num_objects":602,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":602,"num_whiteouts":0,"num_read":15246,"num_read_kb":3072246,"num_write":422204,"num_write_kb":32347863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":70,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588086,"ondisk_log_size":588086,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8133900,"kb_used_data":608076,"kb_used_omap":12,"kb_used_meta":7525747,"kb_avail":746840820,"statfs":{"total":773094113280,"available":764764999680,"internally_reserved":0,"allocated":622669824,"data_stored":621228160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7706365488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-172834971,"num_objects":-1095,"num_object_clones":0,"num_object_copies":-3404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1095,"num_whiteouts":0,"num_read":40,"num_read_kb":6703,"num_write":3385,"num_write_kb":53731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002804"},"pg_stats":[{"pgid":"2.1f","version":"27'2272","reported_seq":2310,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:35.725226+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:26:35.725226+0000","last_peered":"2024-04-25T01:26:35.725226+0000","last_clean":"2024-04-25T01:26:35.725226+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:26:35.725226+0000","last_undegraded":"2024-04-25T01:26:35.725226+0000","last_fullsized":"2024-04-25T01:26:35.725226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2281,"num_write_kb":17945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2538","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.357273+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:27:23.357273+0000","last_peered":"2024-04-25T01:27:23.357273+0000","last_clean":"2024-04-25T01:27:23.357273+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:27:23.357273+0000","last_undegraded":"2024-04-25T01:27:23.357273+0000","last_fullsized":"2024-04-25T01:27:23.357273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2538,"log_dups_size":0,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9425","reported_seq":9743,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.300308+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:27:23.300308+0000","last_peered":"2024-04-25T01:27:23.300308+0000","last_clean":"2024-04-25T01:27:23.300308+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:27:23.300308+0000","last_undegraded":"2024-04-25T01:27:23.300308+0000","last_fullsized":"2024-04-25T01:27:23.300308+0000","mapping_epoch":23,"log_start":"27'3102","ondisk_log_start":"27'3102","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754914,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":290,"num_read_kb":73660,"num_write":6789,"num_write_kb":1006904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6102","reported_seq":6140,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:00.225983+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:27:00.225983+0000","last_peered":"2024-04-25T01:27:00.225983+0000","last_clean":"2024-04-25T01:27:00.225983+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:27:00.225983+0000","last_undegraded":"2024-04-25T01:27:00.225983+0000","last_fullsized":"2024-04-25T01:27:00.225983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6102,"log_dups_size":0,"ondisk_log_size":6102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6110,"num_write_kb":49190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2508","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.936451+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:27:22.936451+0000","last_peered":"2024-04-25T01:27:22.936451+0000","last_clean":"2024-04-25T01:27:22.936451+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:27:22.936451+0000","last_undegraded":"2024-04-25T01:27:22.936451+0000","last_fullsized":"2024-04-25T01:27:22.936451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9551","reported_seq":10008,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.674548+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:27:27.674548+0000","last_peered":"2024-04-25T01:27:27.674548+0000","last_clean":"2024-04-25T01:27:27.674548+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:27:27.674548+0000","last_undegraded":"2024-04-25T01:27:27.674548+0000","last_fullsized":"2024-04-25T01:27:27.674548+0000","mapping_epoch":23,"log_start":"27'2331","ondisk_log_start":"27'2331","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2241862,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":429,"num_read_kb":108966,"num_write":6803,"num_write_kb":1087428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9177","reported_seq":9304,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.354011+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:27:26.354011+0000","last_peered":"2024-04-25T01:27:26.354011+0000","last_clean":"2024-04-25T01:27:26.354011+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:27:26.354011+0000","last_undegraded":"2024-04-25T01:27:26.354011+0000","last_fullsized":"2024-04-25T01:27:26.354011+0000","mapping_epoch":18,"log_start":"27'2200","ondisk_log_start":"27'2200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6977,"log_dups_size":0,"ondisk_log_size":6977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4929315,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":772,"num_write":9170,"num_write_kb":76431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2627","reported_seq":2661,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.334468+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:27:28.334468+0000","last_peered":"2024-04-25T01:27:28.334468+0000","last_clean":"2024-04-25T01:27:28.334468+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:27:28.334468+0000","last_undegraded":"2024-04-25T01:27:28.334468+0000","last_fullsized":"2024-04-25T01:27:28.334468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2627,"log_dups_size":0,"ondisk_log_size":2627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9134","reported_seq":9750,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.567004+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:27:27.567004+0000","last_peered":"2024-04-25T01:27:27.567004+0000","last_clean":"2024-04-25T01:27:27.567004+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:27:27.567004+0000","last_undegraded":"2024-04-25T01:27:27.567004+0000","last_fullsized":"2024-04-25T01:27:27.567004+0000","mapping_epoch":23,"log_start":"27'1914","ondisk_log_start":"27'1914","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3266670,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6424,"num_write_kb":913615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4331","reported_seq":4367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.374958+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:27:26.374958+0000","last_peered":"2024-04-25T01:27:26.374958+0000","last_clean":"2024-04-25T01:27:26.374958+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:27:26.374958+0000","last_undegraded":"2024-04-25T01:27:26.374958+0000","last_fullsized":"2024-04-25T01:27:26.374958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4333,"num_write_kb":43749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2555","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.310340+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:27:28.310340+0000","last_peered":"2024-04-25T01:27:28.310340+0000","last_clean":"2024-04-25T01:27:28.310340+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:27:28.310340+0000","last_undegraded":"2024-04-25T01:27:28.310340+0000","last_fullsized":"2024-04-25T01:27:28.310340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8828","reported_seq":9239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.693327+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:27:27.693327+0000","last_peered":"2024-04-25T01:27:27.693327+0000","last_clean":"2024-04-25T01:27:27.693327+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:27:27.693327+0000","last_undegraded":"2024-04-25T01:27:27.693327+0000","last_fullsized":"2024-04-25T01:27:27.693327+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1257244,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":383,"num_read_kb":97282,"num_write":6126,"num_write_kb":837257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2548","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.095129+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:27:23.095129+0000","last_peered":"2024-04-25T01:27:23.095129+0000","last_clean":"2024-04-25T01:27:23.095129+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:27:23.095129+0000","last_undegraded":"2024-04-25T01:27:23.095129+0000","last_fullsized":"2024-04-25T01:27:23.095129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6999","reported_seq":7036,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.366224+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:27:26.366224+0000","last_peered":"2024-04-25T01:27:26.366224+0000","last_clean":"2024-04-25T01:27:26.366224+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:27:26.366224+0000","last_undegraded":"2024-04-25T01:27:26.366224+0000","last_fullsized":"2024-04-25T01:27:26.366224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6999,"log_dups_size":0,"ondisk_log_size":6999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6999,"num_write_kb":53860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9950","reported_seq":10572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.681856+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:27:27.681856+0000","last_peered":"2024-04-25T01:27:27.681856+0000","last_clean":"2024-04-25T01:27:27.681856+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:27:27.681856+0000","last_undegraded":"2024-04-25T01:27:27.681856+0000","last_fullsized":"2024-04-25T01:27:27.681856+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1888996,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":593,"num_read_kb":150622,"num_write":7196,"num_write_kb":1017659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2562","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.956703+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:27:22.956703+0000","last_peered":"2024-04-25T01:27:22.956703+0000","last_clean":"2024-04-25T01:27:22.956703+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:27:22.956703+0000","last_undegraded":"2024-04-25T01:27:22.956703+0000","last_fullsized":"2024-04-25T01:27:22.956703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7175","reported_seq":7212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.389762+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:27:26.389762+0000","last_peered":"2024-04-25T01:27:26.389762+0000","last_clean":"2024-04-25T01:27:26.389762+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:27:26.389762+0000","last_undegraded":"2024-04-25T01:27:26.389762+0000","last_fullsized":"2024-04-25T01:27:26.389762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7178,"num_write_kb":58045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9175","reported_seq":9667,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.692326+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:27:27.692326+0000","last_peered":"2024-04-25T01:27:27.692326+0000","last_clean":"2024-04-25T01:27:27.692326+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:27:27.692326+0000","last_undegraded":"2024-04-25T01:27:27.692326+0000","last_fullsized":"2024-04-25T01:27:27.692326+0000","mapping_epoch":23,"log_start":"27'1927","ondisk_log_start":"27'1927","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7248,"log_dups_size":0,"ondisk_log_size":7248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1152186,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":467,"num_read_kb":118618,"num_write":6473,"num_write_kb":966622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2628","reported_seq":2663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.092732+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:27:23.092732+0000","last_peered":"2024-04-25T01:27:23.092732+0000","last_clean":"2024-04-25T01:27:23.092732+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:27:23.092732+0000","last_undegraded":"2024-04-25T01:27:23.092732+0000","last_fullsized":"2024-04-25T01:27:23.092732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2628,"log_dups_size":0,"ondisk_log_size":2628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3548","reported_seq":3585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.339055+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:27:26.339055+0000","last_peered":"2024-04-25T01:27:26.339055+0000","last_clean":"2024-04-25T01:27:26.339055+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:27:26.339055+0000","last_undegraded":"2024-04-25T01:27:26.339055+0000","last_fullsized":"2024-04-25T01:27:26.339055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":26965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'8958","reported_seq":9474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.403662+0000","last_change":"2024-04-25T01:02:13.748397+0000","last_active":"2024-04-25T01:27:23.403662+0000","last_peered":"2024-04-25T01:27:23.403662+0000","last_clean":"2024-04-25T01:27:23.403662+0000","last_became_active":"2024-04-25T01:02:13.748037+0000","last_became_peered":"2024-04-25T01:02:13.748037+0000","last_unstale":"2024-04-25T01:27:23.403662+0000","last_undegraded":"2024-04-25T01:27:23.403662+0000","last_fullsized":"2024-04-25T01:27:23.403662+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":8958,"log_dups_size":0,"ondisk_log_size":8958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":616768,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6393,"num_write_kb":892388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2527","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.249690+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:27:23.249690+0000","last_peered":"2024-04-25T01:27:23.249690+0000","last_clean":"2024-04-25T01:27:23.249690+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:27:23.249690+0000","last_undegraded":"2024-04-25T01:27:23.249690+0000","last_fullsized":"2024-04-25T01:27:23.249690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2989","reported_seq":3026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.361419+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:27:26.361419+0000","last_peered":"2024-04-25T01:27:26.361419+0000","last_clean":"2024-04-25T01:27:26.361419+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:27:26.361419+0000","last_undegraded":"2024-04-25T01:27:26.361419+0000","last_fullsized":"2024-04-25T01:27:26.361419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2989,"log_dups_size":0,"ondisk_log_size":2989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2993,"num_write_kb":22443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9521","reported_seq":9738,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.674466+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:27:27.674466+0000","last_peered":"2024-04-25T01:27:27.674466+0000","last_clean":"2024-04-25T01:27:27.674466+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:27:27.674466+0000","last_undegraded":"2024-04-25T01:27:27.674466+0000","last_fullsized":"2024-04-25T01:27:27.674466+0000","mapping_epoch":23,"log_start":"27'2346","ondisk_log_start":"27'2346","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839412,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":193,"num_read_kb":49022,"num_write":6885,"num_write_kb":954026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2491","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.998935+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:27:22.998935+0000","last_peered":"2024-04-25T01:27:22.998935+0000","last_clean":"2024-04-25T01:27:22.998935+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:27:22.998935+0000","last_undegraded":"2024-04-25T01:27:22.998935+0000","last_fullsized":"2024-04-25T01:27:22.998935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2491,"log_dups_size":0,"ondisk_log_size":2491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4621","reported_seq":4658,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:40.881225+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:26:40.881225+0000","last_peered":"2024-04-25T01:26:40.881225+0000","last_clean":"2024-04-25T01:26:40.881225+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:26:40.881225+0000","last_undegraded":"2024-04-25T01:26:40.881225+0000","last_fullsized":"2024-04-25T01:26:40.881225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":39981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9238","reported_seq":9305,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.523276+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:27:27.523276+0000","last_peered":"2024-04-25T01:27:27.523276+0000","last_clean":"2024-04-25T01:27:27.523276+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:27:27.523276+0000","last_undegraded":"2024-04-25T01:27:27.523276+0000","last_fullsized":"2024-04-25T01:27:27.523276+0000","mapping_epoch":23,"log_start":"27'2005","ondisk_log_start":"27'2005","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820727,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":37,"num_read_kb":9398,"num_write":6593,"num_write_kb":862411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2484","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.960479+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:27:22.960479+0000","last_peered":"2024-04-25T01:27:22.960479+0000","last_clean":"2024-04-25T01:27:22.960479+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:27:22.960479+0000","last_undegraded":"2024-04-25T01:27:22.960479+0000","last_fullsized":"2024-04-25T01:27:22.960479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2484,"log_dups_size":0,"ondisk_log_size":2484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6901","reported_seq":7601,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.390267+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:27:26.390267+0000","last_peered":"2024-04-25T01:27:26.390267+0000","last_clean":"2024-04-25T01:27:26.390267+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:27:26.390267+0000","last_undegraded":"2024-04-25T01:27:26.390267+0000","last_fullsized":"2024-04-25T01:27:26.390267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":663,"num_read_kb":4517,"num_write":9702,"num_write_kb":61055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8977","reported_seq":9049,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.399133+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:27:23.399133+0000","last_peered":"2024-04-25T01:27:23.399133+0000","last_clean":"2024-04-25T01:27:23.399133+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:27:23.399133+0000","last_undegraded":"2024-04-25T01:27:23.399133+0000","last_fullsized":"2024-04-25T01:27:23.399133+0000","mapping_epoch":23,"log_start":"27'2722","ondisk_log_start":"27'2722","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2121035,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":10668,"num_write":6308,"num_write_kb":892747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2580","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.405166+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:27:23.405166+0000","last_peered":"2024-04-25T01:27:23.405166+0000","last_clean":"2024-04-25T01:27:23.405166+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:27:23.405166+0000","last_undegraded":"2024-04-25T01:27:23.405166+0000","last_fullsized":"2024-04-25T01:27:23.405166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6064","reported_seq":6102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322120+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:27:26.322120+0000","last_peered":"2024-04-25T01:27:26.322120+0000","last_clean":"2024-04-25T01:27:26.322120+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:27:26.322120+0000","last_undegraded":"2024-04-25T01:27:26.322120+0000","last_fullsized":"2024-04-25T01:27:26.322120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6062,"num_write_kb":49091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9783","reported_seq":10295,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.349957+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:27:23.349957+0000","last_peered":"2024-04-25T01:27:23.349957+0000","last_clean":"2024-04-25T01:27:23.349957+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:27:23.349957+0000","last_undegraded":"2024-04-25T01:27:23.349957+0000","last_fullsized":"2024-04-25T01:27:23.349957+0000","mapping_epoch":23,"log_start":"27'2453","ondisk_log_start":"27'2453","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2380367,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":482,"num_read_kb":122428,"num_write":7046,"num_write_kb":849139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2499","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.314796+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:27:28.314796+0000","last_peered":"2024-04-25T01:27:28.314796+0000","last_clean":"2024-04-25T01:27:28.314796+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:27:28.314796+0000","last_undegraded":"2024-04-25T01:27:28.314796+0000","last_fullsized":"2024-04-25T01:27:28.314796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7885","reported_seq":7923,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:11.111824+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:27:11.111824+0000","last_peered":"2024-04-25T01:27:11.111824+0000","last_clean":"2024-04-25T01:27:11.111824+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:27:11.111824+0000","last_undegraded":"2024-04-25T01:27:11.111824+0000","last_fullsized":"2024-04-25T01:27:11.111824+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7873,"num_write_kb":70555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9266","reported_seq":9744,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.927730+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:27:27.927730+0000","last_peered":"2024-04-25T01:27:27.927730+0000","last_clean":"2024-04-25T01:27:27.927730+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:27:27.927730+0000","last_undegraded":"2024-04-25T01:27:27.927730+0000","last_fullsized":"2024-04-25T01:27:27.927730+0000","mapping_epoch":23,"log_start":"27'1900","ondisk_log_start":"27'1900","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2553648,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":460,"num_read_kb":116840,"num_write":6586,"num_write_kb":921401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2522","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.480449+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:27:23.480449+0000","last_peered":"2024-04-25T01:27:23.480449+0000","last_clean":"2024-04-25T01:27:23.480449+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:27:23.480449+0000","last_undegraded":"2024-04-25T01:27:23.480449+0000","last_fullsized":"2024-04-25T01:27:23.480449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8860","reported_seq":8898,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.115869+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:27:28.115869+0000","last_peered":"2024-04-25T01:27:28.115869+0000","last_clean":"2024-04-25T01:27:28.115869+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:27:28.115869+0000","last_undegraded":"2024-04-25T01:27:28.115869+0000","last_fullsized":"2024-04-25T01:27:28.115869+0000","mapping_epoch":18,"log_start":"27'1500","ondisk_log_start":"27'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7360,"log_dups_size":0,"ondisk_log_size":7360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7300960,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8857,"num_write_kb":62366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9899","reported_seq":10045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.832406+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:27:27.832406+0000","last_peered":"2024-04-25T01:27:27.832406+0000","last_clean":"2024-04-25T01:27:27.832406+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:27:27.832406+0000","last_undegraded":"2024-04-25T01:27:27.832406+0000","last_fullsized":"2024-04-25T01:27:27.832406+0000","mapping_epoch":23,"log_start":"27'2523","ondisk_log_start":"27'2523","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7376,"log_dups_size":0,"ondisk_log_size":7376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071863,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":124,"num_read_kb":31496,"num_write":7205,"num_write_kb":913227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.249751+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:27:23.249751+0000","last_peered":"2024-04-25T01:27:23.249751+0000","last_clean":"2024-04-25T01:27:23.249751+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:27:23.249751+0000","last_undegraded":"2024-04-25T01:27:23.249751+0000","last_fullsized":"2024-04-25T01:27:23.249751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7133","reported_seq":8083,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:00.219862+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:27:00.219862+0000","last_peered":"2024-04-25T01:27:00.219862+0000","last_clean":"2024-04-25T01:27:00.219862+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:27:00.219862+0000","last_undegraded":"2024-04-25T01:27:00.219862+0000","last_fullsized":"2024-04-25T01:27:00.219862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7133,"log_dups_size":0,"ondisk_log_size":7133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":912,"num_read_kb":4677,"num_write":7121,"num_write_kb":61966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9048","reported_seq":9471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.972691+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:27:27.972691+0000","last_peered":"2024-04-25T01:27:27.972691+0000","last_clean":"2024-04-25T01:27:27.972691+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:27:27.972691+0000","last_undegraded":"2024-04-25T01:27:27.972691+0000","last_fullsized":"2024-04-25T01:27:27.972691+0000","mapping_epoch":23,"log_start":"27'1700","ondisk_log_start":"27'1700","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7348,"log_dups_size":0,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4129696,"num_objects":18,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":396,"num_read_kb":100584,"num_write":6310,"num_write_kb":903694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2472","reported_seq":2508,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.104533+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:27:23.104533+0000","last_peered":"2024-04-25T01:27:23.104533+0000","last_clean":"2024-04-25T01:27:23.104533+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:27:23.104533+0000","last_undegraded":"2024-04-25T01:27:23.104533+0000","last_fullsized":"2024-04-25T01:27:23.104533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'6806","reported_seq":6844,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.391028+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:27:26.391028+0000","last_peered":"2024-04-25T01:27:26.391028+0000","last_clean":"2024-04-25T01:27:26.391028+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:27:26.391028+0000","last_undegraded":"2024-04-25T01:27:26.391028+0000","last_fullsized":"2024-04-25T01:27:26.391028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6806,"num_write_kb":57773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9170","reported_seq":9528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.406550+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:27:23.406550+0000","last_peered":"2024-04-25T01:27:23.406550+0000","last_clean":"2024-04-25T01:27:23.406550+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:27:23.406550+0000","last_undegraded":"2024-04-25T01:27:23.406550+0000","last_fullsized":"2024-04-25T01:27:23.406550+0000","mapping_epoch":23,"log_start":"27'1826","ondisk_log_start":"27'1826","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3481570,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":329,"num_read_kb":83566,"num_write":6508,"num_write_kb":1104508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2578","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.129101+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:27:23.129101+0000","last_peered":"2024-04-25T01:27:23.129101+0000","last_clean":"2024-04-25T01:27:23.129101+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:27:23.129101+0000","last_undegraded":"2024-04-25T01:27:23.129101+0000","last_fullsized":"2024-04-25T01:27:23.129101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4347","reported_seq":4385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.334344+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:27:26.334344+0000","last_peered":"2024-04-25T01:27:26.334344+0000","last_clean":"2024-04-25T01:27:26.334344+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:27:26.334344+0000","last_undegraded":"2024-04-25T01:27:26.334344+0000","last_fullsized":"2024-04-25T01:27:26.334344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4347,"num_write_kb":39756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:23:30.725659+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:23:30.725659+0000","last_peered":"2024-04-25T01:23:30.725659+0000","last_clean":"2024-04-25T01:23:30.725659+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:23:30.725659+0000","last_undegraded":"2024-04-25T01:23:30.725659+0000","last_fullsized":"2024-04-25T01:23:30.725659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'9521","reported_seq":9782,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.480947+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:27:23.480947+0000","last_peered":"2024-04-25T01:27:23.480947+0000","last_clean":"2024-04-25T01:27:23.480947+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:27:23.480947+0000","last_undegraded":"2024-04-25T01:27:23.480947+0000","last_fullsized":"2024-04-25T01:27:23.480947+0000","mapping_epoch":23,"log_start":"27'1566","ondisk_log_start":"27'1566","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7955,"log_dups_size":0,"ondisk_log_size":7955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1722100,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":58674,"num_write":6884,"num_write_kb":914721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2510","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.556555+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:27:22.556555+0000","last_peered":"2024-04-25T01:27:22.556555+0000","last_clean":"2024-04-25T01:27:22.556555+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:27:22.556555+0000","last_undegraded":"2024-04-25T01:27:22.556555+0000","last_fullsized":"2024-04-25T01:27:22.556555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7379","reported_seq":7416,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:20.151024+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:26:20.151024+0000","last_peered":"2024-04-25T01:26:20.151024+0000","last_clean":"2024-04-25T01:26:20.151024+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:26:20.151024+0000","last_undegraded":"2024-04-25T01:26:20.151024+0000","last_fullsized":"2024-04-25T01:26:20.151024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7379,"log_dups_size":0,"ondisk_log_size":7379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7384,"num_write_kb":58188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9262","reported_seq":9766,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.128091+0000","last_change":"2024-04-25T01:02:13.830001+0000","last_active":"2024-04-25T01:27:23.128091+0000","last_peered":"2024-04-25T01:27:23.128091+0000","last_clean":"2024-04-25T01:27:23.128091+0000","last_became_active":"2024-04-25T01:02:13.829268+0000","last_became_peered":"2024-04-25T01:02:13.829268+0000","last_unstale":"2024-04-25T01:27:23.128091+0000","last_undegraded":"2024-04-25T01:27:23.128091+0000","last_fullsized":"2024-04-25T01:27:23.128091+0000","mapping_epoch":26,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7950,"log_dups_size":0,"ondisk_log_size":7950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:35.237193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1912054,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":468,"num_read_kb":118872,"num_write":6583,"num_write_kb":924937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2548","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.300867+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:27:28.300867+0000","last_peered":"2024-04-25T01:27:28.300867+0000","last_clean":"2024-04-25T01:27:28.300867+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:27:28.300867+0000","last_undegraded":"2024-04-25T01:27:28.300867+0000","last_fullsized":"2024-04-25T01:27:28.300867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7034","reported_seq":7071,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.339472+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:27:26.339472+0000","last_peered":"2024-04-25T01:27:26.339472+0000","last_clean":"2024-04-25T01:27:26.339472+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:27:26.339472+0000","last_undegraded":"2024-04-25T01:27:26.339472+0000","last_fullsized":"2024-04-25T01:27:26.339472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7035,"num_write_kb":58075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9439","reported_seq":10132,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.394196+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:27:23.394196+0000","last_peered":"2024-04-25T01:27:23.394196+0000","last_clean":"2024-04-25T01:27:23.394196+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:27:23.394196+0000","last_undegraded":"2024-04-25T01:27:23.394196+0000","last_fullsized":"2024-04-25T01:27:23.394196+0000","mapping_epoch":23,"log_start":"27'2038","ondisk_log_start":"27'2038","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7401,"log_dups_size":0,"ondisk_log_size":7401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3448832,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":664,"num_read_kb":168656,"num_write":6687,"num_write_kb":894893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2407","reported_seq":2442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.289927+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:27:23.289927+0000","last_peered":"2024-04-25T01:27:23.289927+0000","last_clean":"2024-04-25T01:27:23.289927+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:27:23.289927+0000","last_undegraded":"2024-04-25T01:27:23.289927+0000","last_fullsized":"2024-04-25T01:27:23.289927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2407,"log_dups_size":0,"ondisk_log_size":2407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8734","reported_seq":8771,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:37.183850+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:26:37.183850+0000","last_peered":"2024-04-25T01:26:37.183850+0000","last_clean":"2024-04-25T01:26:37.183850+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:26:37.183850+0000","last_undegraded":"2024-04-25T01:26:37.183850+0000","last_fullsized":"2024-04-25T01:26:37.183850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8734,"log_dups_size":0,"ondisk_log_size":8734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8742,"num_write_kb":79319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9292","reported_seq":10356,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.817906+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:27:27.817906+0000","last_peered":"2024-04-25T01:27:27.817906+0000","last_clean":"2024-04-25T01:27:27.817906+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:27:27.817906+0000","last_undegraded":"2024-04-25T01:27:27.817906+0000","last_fullsized":"2024-04-25T01:27:27.817906+0000","mapping_epoch":23,"log_start":"27'1886","ondisk_log_start":"27'1886","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7406,"log_dups_size":0,"ondisk_log_size":7406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187142,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1035,"num_read_kb":262890,"num_write":6554,"num_write_kb":1048321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.190123+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:27:23.190123+0000","last_peered":"2024-04-25T01:27:23.190123+0000","last_clean":"2024-04-25T01:27:23.190123+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:27:23.190123+0000","last_undegraded":"2024-04-25T01:27:23.190123+0000","last_fullsized":"2024-04-25T01:27:23.190123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5708","reported_seq":5745,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.330594+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:27:26.330594+0000","last_peered":"2024-04-25T01:27:26.330594+0000","last_clean":"2024-04-25T01:27:26.330594+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:27:26.330594+0000","last_undegraded":"2024-04-25T01:27:26.330594+0000","last_fullsized":"2024-04-25T01:27:26.330594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":44693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'8855","reported_seq":9273,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.523452+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:27:27.523452+0000","last_peered":"2024-04-25T01:27:27.523452+0000","last_clean":"2024-04-25T01:27:27.523452+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:27:27.523452+0000","last_undegraded":"2024-04-25T01:27:27.523452+0000","last_fullsized":"2024-04-25T01:27:27.523452+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7245,"log_dups_size":0,"ondisk_log_size":7245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":412999,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":394,"num_read_kb":100076,"num_write":6145,"num_write_kb":854299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2584","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.380863+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:27:23.380863+0000","last_peered":"2024-04-25T01:27:23.380863+0000","last_clean":"2024-04-25T01:27:23.380863+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:27:23.380863+0000","last_undegraded":"2024-04-25T01:27:23.380863+0000","last_fullsized":"2024-04-25T01:27:23.380863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2584,"log_dups_size":0,"ondisk_log_size":2584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'5129","reported_seq":5166,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:15.200879+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:27:15.200879+0000","last_peered":"2024-04-25T01:27:15.200879+0000","last_clean":"2024-04-25T01:27:15.200879+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:27:15.200879+0000","last_undegraded":"2024-04-25T01:27:15.200879+0000","last_fullsized":"2024-04-25T01:27:15.200879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5129,"log_dups_size":0,"ondisk_log_size":5129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":40185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9360","reported_seq":9696,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.479124+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:27:23.479124+0000","last_peered":"2024-04-25T01:27:23.479124+0000","last_clean":"2024-04-25T01:27:23.479124+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:27:23.479124+0000","last_undegraded":"2024-04-25T01:27:23.479124+0000","last_fullsized":"2024-04-25T01:27:23.479124+0000","mapping_epoch":23,"log_start":"27'2023","ondisk_log_start":"27'2023","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6280246,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":308,"num_read_kb":78232,"num_write":6676,"num_write_kb":1180887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2450","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.397998+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:27:23.397998+0000","last_peered":"2024-04-25T01:27:23.397998+0000","last_clean":"2024-04-25T01:27:23.397998+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:27:23.397998+0000","last_undegraded":"2024-04-25T01:27:23.397998+0000","last_fullsized":"2024-04-25T01:27:23.397998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4840","reported_seq":4877,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:14.595711+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:27:14.595711+0000","last_peered":"2024-04-25T01:27:14.595711+0000","last_clean":"2024-04-25T01:27:14.595711+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:27:14.595711+0000","last_undegraded":"2024-04-25T01:27:14.595711+0000","last_fullsized":"2024-04-25T01:27:14.595711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4844,"num_write_kb":44079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9089","reported_seq":9590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.255740+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:27:23.255740+0000","last_peered":"2024-04-25T01:27:23.255740+0000","last_clean":"2024-04-25T01:27:23.255740+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:27:23.255740+0000","last_undegraded":"2024-04-25T01:27:23.255740+0000","last_fullsized":"2024-04-25T01:27:23.255740+0000","mapping_epoch":23,"log_start":"27'2828","ondisk_log_start":"27'2828","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2716911,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":473,"num_read_kb":120142,"num_write":6358,"num_write_kb":909053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2554","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.084782+0000","last_change":"2024-04-25T01:02:13.744191+0000","last_active":"2024-04-25T01:27:23.084782+0000","last_peered":"2024-04-25T01:27:23.084782+0000","last_clean":"2024-04-25T01:27:23.084782+0000","last_became_active":"2024-04-25T01:02:13.743832+0000","last_became_peered":"2024-04-25T01:02:13.743832+0000","last_unstale":"2024-04-25T01:27:23.084782+0000","last_undegraded":"2024-04-25T01:27:23.084782+0000","last_fullsized":"2024-04-25T01:27:23.084782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"27'2887","reported_seq":2924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.345321+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:27:26.345321+0000","last_peered":"2024-04-25T01:27:26.345321+0000","last_clean":"2024-04-25T01:27:26.345321+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:27:26.345321+0000","last_undegraded":"2024-04-25T01:27:26.345321+0000","last_fullsized":"2024-04-25T01:27:26.345321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2887,"log_dups_size":0,"ondisk_log_size":2887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2884,"num_write_kb":30449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9218","reported_seq":9460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.036523+0000","last_change":"2024-04-25T01:02:13.830353+0000","last_active":"2024-04-25T01:27:23.036523+0000","last_peered":"2024-04-25T01:27:23.036523+0000","last_clean":"2024-04-25T01:27:23.036523+0000","last_became_active":"2024-04-25T01:02:13.829847+0000","last_became_peered":"2024-04-25T01:02:13.829847+0000","last_unstale":"2024-04-25T01:27:23.036523+0000","last_undegraded":"2024-04-25T01:27:23.036523+0000","last_fullsized":"2024-04-25T01:27:23.036523+0000","mapping_epoch":26,"log_start":"27'2967","ondisk_log_start":"27'2967","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:00.933599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1903266,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":52578,"num_write":6608,"num_write_kb":922819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2517","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.153861+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:27:23.153861+0000","last_peered":"2024-04-25T01:27:23.153861+0000","last_clean":"2024-04-25T01:27:23.153861+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:27:23.153861+0000","last_undegraded":"2024-04-25T01:27:23.153861+0000","last_fullsized":"2024-04-25T01:27:23.153861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2517,"log_dups_size":0,"ondisk_log_size":2517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6766","reported_seq":6803,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:03.691415+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:27:03.691415+0000","last_peered":"2024-04-25T01:27:03.691415+0000","last_clean":"2024-04-25T01:27:03.691415+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:27:03.691415+0000","last_undegraded":"2024-04-25T01:27:03.691415+0000","last_fullsized":"2024-04-25T01:27:03.691415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6766,"log_dups_size":0,"ondisk_log_size":6766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6774,"num_write_kb":57696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9611","reported_seq":9984,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.485448+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:27:28.485448+0000","last_peered":"2024-04-25T01:27:28.485448+0000","last_clean":"2024-04-25T01:27:28.485448+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:27:28.485448+0000","last_undegraded":"2024-04-25T01:27:28.485448+0000","last_fullsized":"2024-04-25T01:27:28.485448+0000","mapping_epoch":23,"log_start":"27'2214","ondisk_log_start":"27'2214","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2536524,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":349,"num_read_kb":88646,"num_write":6979,"num_write_kb":1079633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2454","reported_seq":2488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.321625+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:27:23.321625+0000","last_peered":"2024-04-25T01:27:23.321625+0000","last_clean":"2024-04-25T01:27:23.321625+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:27:23.321625+0000","last_undegraded":"2024-04-25T01:27:23.321625+0000","last_fullsized":"2024-04-25T01:27:23.321625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2454,"log_dups_size":0,"ondisk_log_size":2454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10280","reported_seq":10909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:15.207946+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:27:15.207946+0000","last_peered":"2024-04-25T01:27:15.207946+0000","last_clean":"2024-04-25T01:27:15.207946+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:27:15.207946+0000","last_undegraded":"2024-04-25T01:27:15.207946+0000","last_fullsized":"2024-04-25T01:27:15.207946+0000","mapping_epoch":18,"log_start":"27'2300","ondisk_log_start":"27'2300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":594,"num_read_kb":4482,"num_write":10274,"num_write_kb":76426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9142","reported_seq":9483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.928290+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:27:27.928290+0000","last_peered":"2024-04-25T01:27:27.928290+0000","last_clean":"2024-04-25T01:27:27.928290+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:27:27.928290+0000","last_undegraded":"2024-04-25T01:27:27.928290+0000","last_fullsized":"2024-04-25T01:27:27.928290+0000","mapping_epoch":23,"log_start":"27'1734","ondisk_log_start":"27'1734","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7408,"log_dups_size":0,"ondisk_log_size":7408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1592989,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":312,"num_read_kb":79248,"num_write":6479,"num_write_kb":938399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2545","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.035246+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:27:23.035246+0000","last_peered":"2024-04-25T01:27:23.035246+0000","last_clean":"2024-04-25T01:27:23.035246+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:27:23.035246+0000","last_undegraded":"2024-04-25T01:27:23.035246+0000","last_fullsized":"2024-04-25T01:27:23.035246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2545,"log_dups_size":0,"ondisk_log_size":2545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6456","reported_seq":6492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.396435+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:27:26.396435+0000","last_peered":"2024-04-25T01:27:26.396435+0000","last_clean":"2024-04-25T01:27:26.396435+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:27:26.396435+0000","last_undegraded":"2024-04-25T01:27:26.396435+0000","last_fullsized":"2024-04-25T01:27:26.396435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6456,"log_dups_size":0,"ondisk_log_size":6456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6471,"num_write_kb":41578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'8950","reported_seq":9484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.661822+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:27:27.661822+0000","last_peered":"2024-04-25T01:27:27.661822+0000","last_clean":"2024-04-25T01:27:27.661822+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:27:27.661822+0000","last_undegraded":"2024-04-25T01:27:27.661822+0000","last_fullsized":"2024-04-25T01:27:27.661822+0000","mapping_epoch":23,"log_start":"27'1803","ondisk_log_start":"27'1803","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7147,"log_dups_size":0,"ondisk_log_size":7147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2560955,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":508,"num_read_kb":129032,"num_write":6252,"num_write_kb":1000416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2560","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.148657+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:27:23.148657+0000","last_peered":"2024-04-25T01:27:23.148657+0000","last_clean":"2024-04-25T01:27:23.148657+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:27:23.148657+0000","last_undegraded":"2024-04-25T01:27:23.148657+0000","last_fullsized":"2024-04-25T01:27:23.148657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2560,"log_dups_size":0,"ondisk_log_size":2560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6614","reported_seq":6650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.363784+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:27:26.363784+0000","last_peered":"2024-04-25T01:27:26.363784+0000","last_clean":"2024-04-25T01:27:26.363784+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:27:26.363784+0000","last_undegraded":"2024-04-25T01:27:26.363784+0000","last_fullsized":"2024-04-25T01:27:26.363784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6614,"log_dups_size":0,"ondisk_log_size":6614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6608,"num_write_kb":53556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9193","reported_seq":9400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.462589+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:27:23.462589+0000","last_peered":"2024-04-25T01:27:23.462589+0000","last_clean":"2024-04-25T01:27:23.462589+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:27:23.462589+0000","last_undegraded":"2024-04-25T01:27:23.462589+0000","last_fullsized":"2024-04-25T01:27:23.462589+0000","mapping_epoch":23,"log_start":"27'2910","ondisk_log_start":"27'2910","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1149248,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6520,"num_write_kb":962512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2543","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.312816+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:27:28.312816+0000","last_peered":"2024-04-25T01:27:28.312816+0000","last_clean":"2024-04-25T01:27:28.312816+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:27:28.312816+0000","last_undegraded":"2024-04-25T01:27:28.312816+0000","last_fullsized":"2024-04-25T01:27:28.312816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2543,"log_dups_size":0,"ondisk_log_size":2543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6620","reported_seq":7475,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:25:50.038282+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:25:50.038282+0000","last_peered":"2024-04-25T01:25:50.038282+0000","last_clean":"2024-04-25T01:25:50.038282+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:25:50.038282+0000","last_undegraded":"2024-04-25T01:25:50.038282+0000","last_fullsized":"2024-04-25T01:25:50.038282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":821,"num_read_kb":4618,"num_write":6622,"num_write_kb":49469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9979","reported_seq":10043,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.444337+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:27:23.444337+0000","last_peered":"2024-04-25T01:27:23.444337+0000","last_clean":"2024-04-25T01:27:23.444337+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:27:23.444337+0000","last_undegraded":"2024-04-25T01:27:23.444337+0000","last_fullsized":"2024-04-25T01:27:23.444337+0000","mapping_epoch":23,"log_start":"27'3717","ondisk_log_start":"27'3717","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1707840,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":35,"num_read_kb":8890,"num_write":7259,"num_write_kb":1063948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9051","reported_seq":9308,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.341237+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:27:23.341237+0000","last_peered":"2024-04-25T01:27:23.341237+0000","last_clean":"2024-04-25T01:27:23.341237+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:27:23.341237+0000","last_undegraded":"2024-04-25T01:27:23.341237+0000","last_fullsized":"2024-04-25T01:27:23.341237+0000","mapping_epoch":23,"log_start":"27'1719","ondisk_log_start":"27'1719","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":227,"num_read_kb":57658,"num_write":6452,"num_write_kb":909285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2475","reported_seq":2509,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.998206+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:27:22.998206+0000","last_peered":"2024-04-25T01:27:22.998206+0000","last_clean":"2024-04-25T01:27:22.998206+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:27:22.998206+0000","last_undegraded":"2024-04-25T01:27:22.998206+0000","last_fullsized":"2024-04-25T01:27:22.998206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9899","reported_seq":9935,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322490+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:27:26.322490+0000","last_peered":"2024-04-25T01:27:26.322490+0000","last_clean":"2024-04-25T01:27:26.322490+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:27:26.322490+0000","last_undegraded":"2024-04-25T01:27:26.322490+0000","last_fullsized":"2024-04-25T01:27:26.322490+0000","mapping_epoch":18,"log_start":"27'2500","ondisk_log_start":"27'2500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":68120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8927","reported_seq":9786,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.374438+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:27:23.374438+0000","last_peered":"2024-04-25T01:27:23.374438+0000","last_clean":"2024-04-25T01:27:23.374438+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:27:23.374438+0000","last_undegraded":"2024-04-25T01:27:23.374438+0000","last_fullsized":"2024-04-25T01:27:23.374438+0000","mapping_epoch":23,"log_start":"27'1000","ondisk_log_start":"27'1000","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7927,"log_dups_size":0,"ondisk_log_size":7927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2619556,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":829,"num_read_kb":210566,"num_write":6293,"num_write_kb":892044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2577","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.071456+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:27:23.071456+0000","last_peered":"2024-04-25T01:27:23.071456+0000","last_clean":"2024-04-25T01:27:23.071456+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:27:23.071456+0000","last_undegraded":"2024-04-25T01:27:23.071456+0000","last_fullsized":"2024-04-25T01:27:23.071456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5341","reported_seq":5377,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:13.925537+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:26:13.925537+0000","last_peered":"2024-04-25T01:26:13.925537+0000","last_clean":"2024-04-25T01:26:13.925537+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:26:13.925537+0000","last_undegraded":"2024-04-25T01:26:13.925537+0000","last_fullsized":"2024-04-25T01:26:13.925537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5341,"num_write_kb":44512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9496","reported_seq":9532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:14.123973+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:26:14.123973+0000","last_peered":"2024-04-25T01:26:14.123973+0000","last_clean":"2024-04-25T01:26:14.123973+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:26:14.123973+0000","last_undegraded":"2024-04-25T01:26:14.123973+0000","last_fullsized":"2024-04-25T01:26:14.123973+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":9507,"num_write_kb":71968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2515","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.271703+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:27:23.271703+0000","last_peered":"2024-04-25T01:27:23.271703+0000","last_clean":"2024-04-25T01:27:23.271703+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:27:23.271703+0000","last_undegraded":"2024-04-25T01:27:23.271703+0000","last_fullsized":"2024-04-25T01:27:23.271703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9444","reported_seq":9599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.419393+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:27:23.419393+0000","last_peered":"2024-04-25T01:27:23.419393+0000","last_clean":"2024-04-25T01:27:23.419393+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:27:23.419393+0000","last_undegraded":"2024-04-25T01:27:23.419393+0000","last_fullsized":"2024-04-25T01:27:23.419393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4296704,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":127,"num_read_kb":32258,"num_write":6733,"num_write_kb":1000281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8023","reported_seq":8059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:24:51.994967+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:24:51.994967+0000","last_peered":"2024-04-25T01:24:51.994967+0000","last_clean":"2024-04-25T01:24:51.994967+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:24:51.994967+0000","last_undegraded":"2024-04-25T01:24:51.994967+0000","last_fullsized":"2024-04-25T01:24:51.994967+0000","mapping_epoch":18,"log_start":"27'1700","ondisk_log_start":"27'1700","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8018,"num_write_kb":66584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2558","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.048774+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:27:23.048774+0000","last_peered":"2024-04-25T01:27:23.048774+0000","last_clean":"2024-04-25T01:27:23.048774+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:27:23.048774+0000","last_undegraded":"2024-04-25T01:27:23.048774+0000","last_fullsized":"2024-04-25T01:27:23.048774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9429","reported_seq":9825,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.252529+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:27:23.252529+0000","last_peered":"2024-04-25T01:27:23.252529+0000","last_clean":"2024-04-25T01:27:23.252529+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:27:23.252529+0000","last_undegraded":"2024-04-25T01:27:23.252529+0000","last_fullsized":"2024-04-25T01:27:23.252529+0000","mapping_epoch":23,"log_start":"27'3125","ondisk_log_start":"27'3125","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6304,"log_dups_size":0,"ondisk_log_size":6304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6485238,"num_objects":15,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":368,"num_read_kb":93472,"num_write":6715,"num_write_kb":1049905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2448","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.142813+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:27:23.142813+0000","last_peered":"2024-04-25T01:27:23.142813+0000","last_clean":"2024-04-25T01:27:23.142813+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:27:23.142813+0000","last_undegraded":"2024-04-25T01:27:23.142813+0000","last_fullsized":"2024-04-25T01:27:23.142813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2448,"log_dups_size":0,"ondisk_log_size":2448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2578","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.052048+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:27:23.052048+0000","last_peered":"2024-04-25T01:27:23.052048+0000","last_clean":"2024-04-25T01:27:23.052048+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:27:23.052048+0000","last_undegraded":"2024-04-25T01:27:23.052048+0000","last_fullsized":"2024-04-25T01:27:23.052048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2493","reported_seq":2529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.401417+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:27:28.401417+0000","last_peered":"2024-04-25T01:27:28.401417+0000","last_clean":"2024-04-25T01:27:28.401417+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:27:28.401417+0000","last_undegraded":"2024-04-25T01:27:28.401417+0000","last_fullsized":"2024-04-25T01:27:28.401417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2562","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.122836+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:27:23.122836+0000","last_peered":"2024-04-25T01:27:23.122836+0000","last_clean":"2024-04-25T01:27:23.122836+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:27:23.122836+0000","last_undegraded":"2024-04-25T01:27:23.122836+0000","last_fullsized":"2024-04-25T01:27:23.122836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.340997+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:27:23.340997+0000","last_peered":"2024-04-25T01:27:23.340997+0000","last_clean":"2024-04-25T01:27:23.340997+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:27:23.340997+0000","last_undegraded":"2024-04-25T01:27:23.340997+0000","last_fullsized":"2024-04-25T01:27:23.340997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2445","reported_seq":2480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.369622+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:27:23.369622+0000","last_peered":"2024-04-25T01:27:23.369622+0000","last_clean":"2024-04-25T01:27:23.369622+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:27:23.369622+0000","last_undegraded":"2024-04-25T01:27:23.369622+0000","last_fullsized":"2024-04-25T01:27:23.369622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2573","reported_seq":2608,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.402625+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:27:23.402625+0000","last_peered":"2024-04-25T01:27:23.402625+0000","last_clean":"2024-04-25T01:27:23.402625+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:27:23.402625+0000","last_undegraded":"2024-04-25T01:27:23.402625+0000","last_fullsized":"2024-04-25T01:27:23.402625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2538","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.170149+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:27:23.170149+0000","last_peered":"2024-04-25T01:27:23.170149+0000","last_clean":"2024-04-25T01:27:23.170149+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:27:23.170149+0000","last_undegraded":"2024-04-25T01:27:23.170149+0000","last_fullsized":"2024-04-25T01:27:23.170149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2538,"log_dups_size":0,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2529","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.289774+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:27:28.289774+0000","last_peered":"2024-04-25T01:27:28.289774+0000","last_clean":"2024-04-25T01:27:28.289774+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:27:28.289774+0000","last_undegraded":"2024-04-25T01:27:28.289774+0000","last_fullsized":"2024-04-25T01:27:28.289774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2500","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.365002+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:27:23.365002+0000","last_peered":"2024-04-25T01:27:23.365002+0000","last_clean":"2024-04-25T01:27:23.365002+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:27:23.365002+0000","last_undegraded":"2024-04-25T01:27:23.365002+0000","last_fullsized":"2024-04-25T01:27:23.365002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2557","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.475740+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:27:28.475740+0000","last_peered":"2024-04-25T01:27:28.475740+0000","last_clean":"2024-04-25T01:27:28.475740+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:27:28.475740+0000","last_undegraded":"2024-04-25T01:27:28.475740+0000","last_fullsized":"2024-04-25T01:27:28.475740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2567","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.273548+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:27:23.273548+0000","last_peered":"2024-04-25T01:27:23.273548+0000","last_clean":"2024-04-25T01:27:23.273548+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:27:23.273548+0000","last_undegraded":"2024-04-25T01:27:23.273548+0000","last_fullsized":"2024-04-25T01:27:23.273548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2572","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.152624+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:27:23.152624+0000","last_peered":"2024-04-25T01:27:23.152624+0000","last_clean":"2024-04-25T01:27:23.152624+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:27:23.152624+0000","last_undegraded":"2024-04-25T01:27:23.152624+0000","last_fullsized":"2024-04-25T01:27:23.152624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2572,"log_dups_size":0,"ondisk_log_size":2572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2573","reported_seq":2608,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.393729+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:27:23.393729+0000","last_peered":"2024-04-25T01:27:23.393729+0000","last_clean":"2024-04-25T01:27:23.393729+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:27:23.393729+0000","last_undegraded":"2024-04-25T01:27:23.393729+0000","last_fullsized":"2024-04-25T01:27:23.393729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2512","reported_seq":2547,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.365445+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:27:23.365445+0000","last_peered":"2024-04-25T01:27:23.365445+0000","last_clean":"2024-04-25T01:27:23.365445+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:27:23.365445+0000","last_undegraded":"2024-04-25T01:27:23.365445+0000","last_fullsized":"2024-04-25T01:27:23.365445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2512,"log_dups_size":0,"ondisk_log_size":2512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2588","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.349289+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:27:28.349289+0000","last_peered":"2024-04-25T01:27:28.349289+0000","last_clean":"2024-04-25T01:27:28.349289+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:27:28.349289+0000","last_undegraded":"2024-04-25T01:27:28.349289+0000","last_fullsized":"2024-04-25T01:27:28.349289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2509","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.305277+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:27:28.305277+0000","last_peered":"2024-04-25T01:27:28.305277+0000","last_clean":"2024-04-25T01:27:28.305277+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:27:28.305277+0000","last_undegraded":"2024-04-25T01:27:28.305277+0000","last_fullsized":"2024-04-25T01:27:28.305277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.045603+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:27:23.045603+0000","last_peered":"2024-04-25T01:27:23.045603+0000","last_clean":"2024-04-25T01:27:23.045603+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:27:23.045603+0000","last_undegraded":"2024-04-25T01:27:23.045603+0000","last_fullsized":"2024-04-25T01:27:23.045603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2482","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.837016+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:27:22.837016+0000","last_peered":"2024-04-25T01:27:22.837016+0000","last_clean":"2024-04-25T01:27:22.837016+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:27:22.837016+0000","last_undegraded":"2024-04-25T01:27:22.837016+0000","last_fullsized":"2024-04-25T01:27:22.837016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2471","reported_seq":2505,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.182821+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:27:23.182821+0000","last_peered":"2024-04-25T01:27:23.182821+0000","last_clean":"2024-04-25T01:27:23.182821+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:27:23.182821+0000","last_undegraded":"2024-04-25T01:27:23.182821+0000","last_fullsized":"2024-04-25T01:27:23.182821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2431","reported_seq":2465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.795020+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:27:22.795020+0000","last_peered":"2024-04-25T01:27:22.795020+0000","last_clean":"2024-04-25T01:27:22.795020+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:27:22.795020+0000","last_undegraded":"2024-04-25T01:27:22.795020+0000","last_fullsized":"2024-04-25T01:27:22.795020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2431,"log_dups_size":0,"ondisk_log_size":2431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2464","reported_seq":2503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.288941+0000","last_change":"2024-04-25T01:02:13.749183+0000","last_active":"2024-04-25T01:27:23.288941+0000","last_peered":"2024-04-25T01:27:23.288941+0000","last_clean":"2024-04-25T01:27:23.288941+0000","last_became_active":"2024-04-25T01:02:13.748901+0000","last_became_peered":"2024-04-25T01:02:13.748901+0000","last_unstale":"2024-04-25T01:27:23.288941+0000","last_undegraded":"2024-04-25T01:27:23.288941+0000","last_fullsized":"2024-04-25T01:27:23.288941+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2464,"log_dups_size":0,"ondisk_log_size":2464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:54.284115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2596","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.443677+0000","last_change":"2024-04-25T01:02:13.750135+0000","last_active":"2024-04-25T01:27:23.443677+0000","last_peered":"2024-04-25T01:27:23.443677+0000","last_clean":"2024-04-25T01:27:23.443677+0000","last_became_active":"2024-04-25T01:02:13.749859+0000","last_became_peered":"2024-04-25T01:02:13.749859+0000","last_unstale":"2024-04-25T01:27:23.443677+0000","last_undegraded":"2024-04-25T01:27:23.443677+0000","last_fullsized":"2024-04-25T01:27:23.443677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:18.550102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2552","reported_seq":2586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.351165+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:27:23.351165+0000","last_peered":"2024-04-25T01:27:23.351165+0000","last_clean":"2024-04-25T01:27:23.351165+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:27:23.351165+0000","last_undegraded":"2024-04-25T01:27:23.351165+0000","last_fullsized":"2024-04-25T01:27:23.351165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2552,"log_dups_size":0,"ondisk_log_size":2552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2498","reported_seq":2518,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.384142+0000","last_change":"2024-04-25T01:02:13.828931+0000","last_active":"2024-04-25T01:27:23.384142+0000","last_peered":"2024-04-25T01:27:23.384142+0000","last_clean":"2024-04-25T01:27:23.384142+0000","last_became_active":"2024-04-25T01:02:13.828599+0000","last_became_peered":"2024-04-25T01:02:13.828599+0000","last_unstale":"2024-04-25T01:27:23.384142+0000","last_undegraded":"2024-04-25T01:27:23.384142+0000","last_fullsized":"2024-04-25T01:27:23.384142+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:20.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.39","version":"27'2514","reported_seq":2548,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.084091+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:27:23.084091+0000","last_peered":"2024-04-25T01:27:23.084091+0000","last_clean":"2024-04-25T01:27:23.084091+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:27:23.084091+0000","last_undegraded":"2024-04-25T01:27:23.084091+0000","last_fullsized":"2024-04-25T01:27:23.084091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2468","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.359740+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:27:28.359740+0000","last_peered":"2024-04-25T01:27:28.359740+0000","last_clean":"2024-04-25T01:27:28.359740+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:27:28.359740+0000","last_undegraded":"2024-04-25T01:27:28.359740+0000","last_fullsized":"2024-04-25T01:27:28.359740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2468,"log_dups_size":0,"ondisk_log_size":2468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2492","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.629304+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:27:22.629304+0000","last_peered":"2024-04-25T01:27:22.629304+0000","last_clean":"2024-04-25T01:27:22.629304+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:27:22.629304+0000","last_undegraded":"2024-04-25T01:27:22.629304+0000","last_fullsized":"2024-04-25T01:27:22.629304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2492,"log_dups_size":0,"ondisk_log_size":2492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2557","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.305843+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:27:28.305843+0000","last_peered":"2024-04-25T01:27:28.305843+0000","last_clean":"2024-04-25T01:27:28.305843+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:27:28.305843+0000","last_undegraded":"2024-04-25T01:27:28.305843+0000","last_fullsized":"2024-04-25T01:27:28.305843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2606","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.217124+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:27:23.217124+0000","last_peered":"2024-04-25T01:27:23.217124+0000","last_clean":"2024-04-25T01:27:23.217124+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:27:23.217124+0000","last_undegraded":"2024-04-25T01:27:23.217124+0000","last_fullsized":"2024-04-25T01:27:23.217124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2458","reported_seq":2494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.190500+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:27:23.190500+0000","last_peered":"2024-04-25T01:27:23.190500+0000","last_clean":"2024-04-25T01:27:23.190500+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:27:23.190500+0000","last_undegraded":"2024-04-25T01:27:23.190500+0000","last_fullsized":"2024-04-25T01:27:23.190500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2434","reported_seq":2470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.399721+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:27:23.399721+0000","last_peered":"2024-04-25T01:27:23.399721+0000","last_clean":"2024-04-25T01:27:23.399721+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:27:23.399721+0000","last_undegraded":"2024-04-25T01:27:23.399721+0000","last_fullsized":"2024-04-25T01:27:23.399721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":78633850,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":12020,"num_read_kb":3053080,"num_write":211822,"num_write_kb":30669379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160497664,"data_stored":160497664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232144,"ondisk_log_size":232144,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161709,"ondisk_log_size":161709,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":230335553,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":3110,"num_read_kb":19066,"num_write":209236,"num_write_kb":1676068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":70,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459538432,"data_stored":459478144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194116,"ondisk_log_size":194116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575165,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":956092,"kb_used_data":58612,"kb_used_omap":1,"kb_used_meta":897470,"kb_avail":93415748,"statfs":{"total":96636764160,"available":95657725952,"internally_reserved":0,"allocated":60018688,"data_stored":59837569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":919009738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1202892,"kb_used_data":89156,"kb_used_omap":1,"kb_used_meta":1113726,"kb_avail":93168948,"statfs":{"total":96636764160,"available":95405002752,"internally_reserved":0,"allocated":91295744,"data_stored":91116082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1140455878},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1009360,"kb_used_data":57864,"kb_used_omap":1,"kb_used_meta":951486,"kb_avail":93362480,"statfs":{"total":96636764160,"available":95603179520,"internally_reserved":0,"allocated":59252736,"data_stored":59074738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":974322117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575162,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1013728,"kb_used_data":80280,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93358112,"statfs":{"total":96636764160,"available":95598706688,"internally_reserved":0,"allocated":82206720,"data_stored":82022774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":955840965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1154908,"kb_used_data":99284,"kb_used_omap":1,"kb_used_meta":1055614,"kb_avail":93216932,"statfs":{"total":96636764160,"available":95454138368,"internally_reserved":0,"allocated":101666816,"data_stored":101487512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1080949191},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694676,"kb_used_data":72140,"kb_used_omap":1,"kb_used_meta":622526,"kb_avail":93677164,"statfs":{"total":96636764160,"available":95925415936,"internally_reserved":0,"allocated":73871360,"data_stored":73699576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637467077},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1251780,"kb_used_data":79612,"kb_used_omap":1,"kb_used_meta":1172158,"kb_avail":93120060,"statfs":{"total":96636764160,"available":95354941440,"internally_reserved":0,"allocated":81522688,"data_stored":81341823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1200290245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850464,"kb_used_data":71128,"kb_used_omap":1,"kb_used_meta":779326,"kb_avail":93521376,"statfs":{"total":96636764160,"available":95765889024,"internally_reserved":0,"allocated":72835072,"data_stored":72648086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":798030277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54538240,"data_stored":54526590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60657664,"data_stored":60648583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54525952,"data_stored":54525952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40861696,"data_stored":40855482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":67837952,"data_stored":67830074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":46878720,"data_stored":46872937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17412096,"data_stored":17412096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26370048,"data_stored":26370048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19054592,"data_stored":19054592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27361280,"data_stored":27361280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17469440,"data_stored":17469440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":17408000,"data_stored":17408000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23166976,"data_stored":23166976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12255232,"data_stored":12255232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:27:29.698 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T01:27:29.699 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:27:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.853+0000 7f7588b70640 1 Processor -- start 2024-04-25T01:27:29.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.853+0000 7f7588b70640 1 -- start start 2024-04-25T01:27:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7588b70640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f75840c5520 0x7f75840ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7588b70640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7588b70640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75840c9860 con 0x7f7584152fa0 2024-04-25T01:27:29.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7588b70640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f75840c5a70 con 0x7f75840c5520 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7588b70640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f75840c5d50 con 0x7f75841561e0 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f757bfff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f75840c5520 0x7f75840ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f758259c640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f7584152fa0 0x7f75840c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:44238/0 (socket says 172.21.15.129:44238) 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f758259c640 1 -- 172.21.15.129:0/3549133299 learned_addr learned my addr 172.21.15.129:0/3549133299 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f757bfff640 1 -- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 msgr2=0x7f75840c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f757bfff640 1 --2- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f757bfff640 1 -- 172.21.15.129:0/3549133299 >> v1:172.21.15.171:6789/0 conn(0x7f7584152fa0 legacy=0x7f75840c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7581d9b640 1 -- 172.21.15.129:0/3549133299 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563664922 0 0) 0x7f7574001110 con 0x7f7584152fa0 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f757bfff640 1 -- 172.21.15.129:0/3549133299 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75840c7650 con 0x7f75840c5520 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.854+0000 7f7582d9d640 1 --2- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.855+0000 7f757bfff640 1 --2- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75840c5520 0x7f75840ca2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097a0 tx=0x7f756c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4aa12ae80e66c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.855+0000 7f7581d9b640 1 -- 172.21.15.129:0/3549133299 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c037070 con 0x7f75840c5520 2024-04-25T01:27:29.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.855+0000 7f7581d9b640 1 -- 172.21.15.129:0/3549133299 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c02adb0 con 0x7f75840c5520 2024-04-25T01:27:29.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.855+0000 7f7581d9b640 1 -- 172.21.15.129:0/3549133299 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c032660 con 0x7f75840c5520 2024-04-25T01:27:29.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 -- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75840c5520 msgr2=0x7f75840ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 --2- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75840c5520 0x7f75840ca2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097a0 tx=0x7f756c02a060 comp rx=0 tx=0).stop 2024-04-25T01:27:29.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 -- 172.21.15.129:0/3549133299 shutdown_connections 2024-04-25T01:27:29.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 --2- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 --2- 172.21.15.129:0/3549133299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75840c5520 0x7f75840ca2e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 -- 172.21.15.129:0/3549133299 >> 172.21.15.129:0/3549133299 conn(0x7f758406c400 msgr2=0x7f758406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 -- 172.21.15.129:0/3549133299 shutdown_connections 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.856+0000 7f7588b70640 1 -- 172.21.15.129:0/3549133299 wait complete. 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.857+0000 7f7588b70640 1 Processor -- start 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.857+0000 7f7588b70640 1 -- start start 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 0x7f758414e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7584152fa0 0x7f75840c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75840c9200 con 0x7f75840c5520 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f75840c6030 con 0x7f7584152fa0 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f75840c9380 con 0x7f75841561e0 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 0x7f758414e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 0x7f758414e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55340/0 (socket says 172.21.15.129:55340) 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 -- 172.21.15.129:0/1272078382 learned_addr learned my addr 172.21.15.129:0/1272078382 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f757bfff640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7584152fa0 0x7f75840c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7582d9d640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 -- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 msgr2=0x7f75840c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 -- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7584152fa0 msgr2=0x7f75840c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7584152fa0 0x7f75840c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 -- 172.21.15.129:0/1272078382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75841eb520 con 0x7f75840c5520 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7582d9d640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f758259c640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 0x7f758414e280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f757400c430 tx=0x7f757400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740074e0 con 0x7f75840c5520 2024-04-25T01:27:29.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740047e0 con 0x7f75840c5520 2024-04-25T01:27:29.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574004bd0 con 0x7f75840c5520 2024-04-25T01:27:29.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75840c42e0 con 0x7f75840c5520 2024-04-25T01:27:29.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.858+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75840c34a0 con 0x7f75840c5520 2024-04-25T01:27:29.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.859+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758406b8a0 con 0x7f75840c5520 2024-04-25T01:27:29.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.860+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7574011070 con 0x7f75840c5520 2024-04-25T01:27:29.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.860+0000 7f757affd640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:29.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.860+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f757408adf0 con 0x7f75840c5520 2024-04-25T01:27:29.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.860+0000 7f757bfff640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:29.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.861+0000 7f757bfff640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f756406be30 0x7f756406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f756c034000 tx=0x7f756c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:29.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:29.868+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f758406b8a0 con 0x7f75840c5520 2024-04-25T01:27:30.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.046+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7584075550 con 0x7f756406be30 2024-04-25T01:27:30.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.054+0000 7f757affd640 1 -- 172.21.15.129:0/1272078382 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336161 (secure 0 0 0) 0x7f7584075550 con 0x7f756406be30 2024-04-25T01:27:30.037 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:30.039 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f756406be30 msgr2=0x7f756406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f756406be30 0x7f756406e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f756c034000 tx=0x7f756c02e000 comp rx=0 tx=0).stop 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 msgr2=0x7f758414e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 0x7f758414e280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f757400c430 tx=0x7f757400c830 comp rx=0 tx=0).stop 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 shutdown_connections 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f75841561e0 0x7f75840c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7584152fa0 0x7f75840c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 --2- 172.21.15.129:0/1272078382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75840c5520 0x7f758414e280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 >> 172.21.15.129:0/1272078382 conn(0x7f758406c400 msgr2=0x7f75840aa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 shutdown_connections 2024-04-25T01:27:30.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.058+0000 7f7588b70640 1 -- 172.21.15.129:0/1272078382 wait complete. 2024-04-25T01:27:30.051 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":805,"stamp":"2024-04-25T01:27:29.057652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":309559771,"num_objects":602,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":602,"num_whiteouts":0,"num_read":15246,"num_read_kb":3072246,"num_write":422204,"num_write_kb":32347863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":70,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588086,"ondisk_log_size":588086,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8133900,"kb_used_data":608076,"kb_used_omap":12,"kb_used_meta":7525747,"kb_avail":746840820,"statfs":{"total":773094113280,"available":764764999680,"internally_reserved":0,"allocated":622669824,"data_stored":621228160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7706365488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-172834971,"num_objects":-1095,"num_object_clones":0,"num_object_copies":-3404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1095,"num_whiteouts":0,"num_read":40,"num_read_kb":6703,"num_write":3385,"num_write_kb":53731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002804"},"pg_stats":[{"pgid":"2.1f","version":"27'2272","reported_seq":2310,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:35.725226+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:26:35.725226+0000","last_peered":"2024-04-25T01:26:35.725226+0000","last_clean":"2024-04-25T01:26:35.725226+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:26:35.725226+0000","last_undegraded":"2024-04-25T01:26:35.725226+0000","last_fullsized":"2024-04-25T01:26:35.725226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2281,"num_write_kb":17945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2538","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.357273+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:27:23.357273+0000","last_peered":"2024-04-25T01:27:23.357273+0000","last_clean":"2024-04-25T01:27:23.357273+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:27:23.357273+0000","last_undegraded":"2024-04-25T01:27:23.357273+0000","last_fullsized":"2024-04-25T01:27:23.357273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2538,"log_dups_size":0,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9425","reported_seq":9743,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.300308+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:27:23.300308+0000","last_peered":"2024-04-25T01:27:23.300308+0000","last_clean":"2024-04-25T01:27:23.300308+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:27:23.300308+0000","last_undegraded":"2024-04-25T01:27:23.300308+0000","last_fullsized":"2024-04-25T01:27:23.300308+0000","mapping_epoch":23,"log_start":"27'3102","ondisk_log_start":"27'3102","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:20.195837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754914,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":290,"num_read_kb":73660,"num_write":6789,"num_write_kb":1006904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6102","reported_seq":6140,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:00.225983+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:27:00.225983+0000","last_peered":"2024-04-25T01:27:00.225983+0000","last_clean":"2024-04-25T01:27:00.225983+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:27:00.225983+0000","last_undegraded":"2024-04-25T01:27:00.225983+0000","last_fullsized":"2024-04-25T01:27:00.225983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6102,"log_dups_size":0,"ondisk_log_size":6102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6110,"num_write_kb":49190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2508","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.936451+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:27:22.936451+0000","last_peered":"2024-04-25T01:27:22.936451+0000","last_clean":"2024-04-25T01:27:22.936451+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:27:22.936451+0000","last_undegraded":"2024-04-25T01:27:22.936451+0000","last_fullsized":"2024-04-25T01:27:22.936451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9551","reported_seq":10008,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.674548+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:27:27.674548+0000","last_peered":"2024-04-25T01:27:27.674548+0000","last_clean":"2024-04-25T01:27:27.674548+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:27:27.674548+0000","last_undegraded":"2024-04-25T01:27:27.674548+0000","last_fullsized":"2024-04-25T01:27:27.674548+0000","mapping_epoch":23,"log_start":"27'2331","ondisk_log_start":"27'2331","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2241862,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":429,"num_read_kb":108966,"num_write":6803,"num_write_kb":1087428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9177","reported_seq":9304,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.354011+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:27:26.354011+0000","last_peered":"2024-04-25T01:27:26.354011+0000","last_clean":"2024-04-25T01:27:26.354011+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:27:26.354011+0000","last_undegraded":"2024-04-25T01:27:26.354011+0000","last_fullsized":"2024-04-25T01:27:26.354011+0000","mapping_epoch":18,"log_start":"27'2200","ondisk_log_start":"27'2200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6977,"log_dups_size":0,"ondisk_log_size":6977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4929315,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":772,"num_write":9170,"num_write_kb":76431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2627","reported_seq":2661,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.334468+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:27:28.334468+0000","last_peered":"2024-04-25T01:27:28.334468+0000","last_clean":"2024-04-25T01:27:28.334468+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:27:28.334468+0000","last_undegraded":"2024-04-25T01:27:28.334468+0000","last_fullsized":"2024-04-25T01:27:28.334468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2627,"log_dups_size":0,"ondisk_log_size":2627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9134","reported_seq":9750,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.567004+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:27:27.567004+0000","last_peered":"2024-04-25T01:27:27.567004+0000","last_clean":"2024-04-25T01:27:27.567004+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:27:27.567004+0000","last_undegraded":"2024-04-25T01:27:27.567004+0000","last_fullsized":"2024-04-25T01:27:27.567004+0000","mapping_epoch":23,"log_start":"27'1914","ondisk_log_start":"27'1914","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3266670,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6424,"num_write_kb":913615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4331","reported_seq":4367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.374958+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:27:26.374958+0000","last_peered":"2024-04-25T01:27:26.374958+0000","last_clean":"2024-04-25T01:27:26.374958+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:27:26.374958+0000","last_undegraded":"2024-04-25T01:27:26.374958+0000","last_fullsized":"2024-04-25T01:27:26.374958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4333,"num_write_kb":43749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2555","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.310340+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:27:28.310340+0000","last_peered":"2024-04-25T01:27:28.310340+0000","last_clean":"2024-04-25T01:27:28.310340+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:27:28.310340+0000","last_undegraded":"2024-04-25T01:27:28.310340+0000","last_fullsized":"2024-04-25T01:27:28.310340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8828","reported_seq":9239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.693327+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:27:27.693327+0000","last_peered":"2024-04-25T01:27:27.693327+0000","last_clean":"2024-04-25T01:27:27.693327+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:27:27.693327+0000","last_undegraded":"2024-04-25T01:27:27.693327+0000","last_fullsized":"2024-04-25T01:27:27.693327+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1257244,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":383,"num_read_kb":97282,"num_write":6126,"num_write_kb":837257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2548","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.095129+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:27:23.095129+0000","last_peered":"2024-04-25T01:27:23.095129+0000","last_clean":"2024-04-25T01:27:23.095129+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:27:23.095129+0000","last_undegraded":"2024-04-25T01:27:23.095129+0000","last_fullsized":"2024-04-25T01:27:23.095129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6999","reported_seq":7036,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.366224+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:27:26.366224+0000","last_peered":"2024-04-25T01:27:26.366224+0000","last_clean":"2024-04-25T01:27:26.366224+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:27:26.366224+0000","last_undegraded":"2024-04-25T01:27:26.366224+0000","last_fullsized":"2024-04-25T01:27:26.366224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6999,"log_dups_size":0,"ondisk_log_size":6999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6999,"num_write_kb":53860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9950","reported_seq":10572,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.681856+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:27:27.681856+0000","last_peered":"2024-04-25T01:27:27.681856+0000","last_clean":"2024-04-25T01:27:27.681856+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:27:27.681856+0000","last_undegraded":"2024-04-25T01:27:27.681856+0000","last_fullsized":"2024-04-25T01:27:27.681856+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7220,"log_dups_size":0,"ondisk_log_size":7220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1888996,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":593,"num_read_kb":150622,"num_write":7196,"num_write_kb":1017659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2562","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.956703+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:27:22.956703+0000","last_peered":"2024-04-25T01:27:22.956703+0000","last_clean":"2024-04-25T01:27:22.956703+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:27:22.956703+0000","last_undegraded":"2024-04-25T01:27:22.956703+0000","last_fullsized":"2024-04-25T01:27:22.956703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7175","reported_seq":7212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.389762+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:27:26.389762+0000","last_peered":"2024-04-25T01:27:26.389762+0000","last_clean":"2024-04-25T01:27:26.389762+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:27:26.389762+0000","last_undegraded":"2024-04-25T01:27:26.389762+0000","last_fullsized":"2024-04-25T01:27:26.389762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7178,"num_write_kb":58045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9175","reported_seq":9667,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.692326+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:27:27.692326+0000","last_peered":"2024-04-25T01:27:27.692326+0000","last_clean":"2024-04-25T01:27:27.692326+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:27:27.692326+0000","last_undegraded":"2024-04-25T01:27:27.692326+0000","last_fullsized":"2024-04-25T01:27:27.692326+0000","mapping_epoch":23,"log_start":"27'1927","ondisk_log_start":"27'1927","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7248,"log_dups_size":0,"ondisk_log_size":7248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1152186,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":467,"num_read_kb":118618,"num_write":6473,"num_write_kb":966622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2628","reported_seq":2663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.092732+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:27:23.092732+0000","last_peered":"2024-04-25T01:27:23.092732+0000","last_clean":"2024-04-25T01:27:23.092732+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:27:23.092732+0000","last_undegraded":"2024-04-25T01:27:23.092732+0000","last_fullsized":"2024-04-25T01:27:23.092732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2628,"log_dups_size":0,"ondisk_log_size":2628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3548","reported_seq":3585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.339055+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:27:26.339055+0000","last_peered":"2024-04-25T01:27:26.339055+0000","last_clean":"2024-04-25T01:27:26.339055+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:27:26.339055+0000","last_undegraded":"2024-04-25T01:27:26.339055+0000","last_fullsized":"2024-04-25T01:27:26.339055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":26965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'8958","reported_seq":9474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.403662+0000","last_change":"2024-04-25T01:02:13.748397+0000","last_active":"2024-04-25T01:27:23.403662+0000","last_peered":"2024-04-25T01:27:23.403662+0000","last_clean":"2024-04-25T01:27:23.403662+0000","last_became_active":"2024-04-25T01:02:13.748037+0000","last_became_peered":"2024-04-25T01:02:13.748037+0000","last_unstale":"2024-04-25T01:27:23.403662+0000","last_undegraded":"2024-04-25T01:27:23.403662+0000","last_fullsized":"2024-04-25T01:27:23.403662+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":8958,"log_dups_size":0,"ondisk_log_size":8958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":616768,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6393,"num_write_kb":892388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2527","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.249690+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:27:23.249690+0000","last_peered":"2024-04-25T01:27:23.249690+0000","last_clean":"2024-04-25T01:27:23.249690+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:27:23.249690+0000","last_undegraded":"2024-04-25T01:27:23.249690+0000","last_fullsized":"2024-04-25T01:27:23.249690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2989","reported_seq":3026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.361419+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:27:26.361419+0000","last_peered":"2024-04-25T01:27:26.361419+0000","last_clean":"2024-04-25T01:27:26.361419+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:27:26.361419+0000","last_undegraded":"2024-04-25T01:27:26.361419+0000","last_fullsized":"2024-04-25T01:27:26.361419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2989,"log_dups_size":0,"ondisk_log_size":2989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2993,"num_write_kb":22443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9521","reported_seq":9738,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.674466+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:27:27.674466+0000","last_peered":"2024-04-25T01:27:27.674466+0000","last_clean":"2024-04-25T01:27:27.674466+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:27:27.674466+0000","last_undegraded":"2024-04-25T01:27:27.674466+0000","last_fullsized":"2024-04-25T01:27:27.674466+0000","mapping_epoch":23,"log_start":"27'2346","ondisk_log_start":"27'2346","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839412,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":193,"num_read_kb":49022,"num_write":6885,"num_write_kb":954026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2491","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.998935+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:27:22.998935+0000","last_peered":"2024-04-25T01:27:22.998935+0000","last_clean":"2024-04-25T01:27:22.998935+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:27:22.998935+0000","last_undegraded":"2024-04-25T01:27:22.998935+0000","last_fullsized":"2024-04-25T01:27:22.998935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2491,"log_dups_size":0,"ondisk_log_size":2491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4621","reported_seq":4658,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:40.881225+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:26:40.881225+0000","last_peered":"2024-04-25T01:26:40.881225+0000","last_clean":"2024-04-25T01:26:40.881225+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:26:40.881225+0000","last_undegraded":"2024-04-25T01:26:40.881225+0000","last_fullsized":"2024-04-25T01:26:40.881225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4621,"log_dups_size":0,"ondisk_log_size":4621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":39981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9238","reported_seq":9305,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.523276+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:27:27.523276+0000","last_peered":"2024-04-25T01:27:27.523276+0000","last_clean":"2024-04-25T01:27:27.523276+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:27:27.523276+0000","last_undegraded":"2024-04-25T01:27:27.523276+0000","last_fullsized":"2024-04-25T01:27:27.523276+0000","mapping_epoch":23,"log_start":"27'2005","ondisk_log_start":"27'2005","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820727,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":37,"num_read_kb":9398,"num_write":6593,"num_write_kb":862411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2484","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.960479+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:27:22.960479+0000","last_peered":"2024-04-25T01:27:22.960479+0000","last_clean":"2024-04-25T01:27:22.960479+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:27:22.960479+0000","last_undegraded":"2024-04-25T01:27:22.960479+0000","last_fullsized":"2024-04-25T01:27:22.960479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2484,"log_dups_size":0,"ondisk_log_size":2484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6901","reported_seq":7601,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.390267+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:27:26.390267+0000","last_peered":"2024-04-25T01:27:26.390267+0000","last_clean":"2024-04-25T01:27:26.390267+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:27:26.390267+0000","last_undegraded":"2024-04-25T01:27:26.390267+0000","last_fullsized":"2024-04-25T01:27:26.390267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":663,"num_read_kb":4517,"num_write":9702,"num_write_kb":61055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'8977","reported_seq":9049,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.399133+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:27:23.399133+0000","last_peered":"2024-04-25T01:27:23.399133+0000","last_clean":"2024-04-25T01:27:23.399133+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:27:23.399133+0000","last_undegraded":"2024-04-25T01:27:23.399133+0000","last_fullsized":"2024-04-25T01:27:23.399133+0000","mapping_epoch":23,"log_start":"27'2722","ondisk_log_start":"27'2722","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2121035,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":10668,"num_write":6308,"num_write_kb":892747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2580","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.405166+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:27:23.405166+0000","last_peered":"2024-04-25T01:27:23.405166+0000","last_clean":"2024-04-25T01:27:23.405166+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:27:23.405166+0000","last_undegraded":"2024-04-25T01:27:23.405166+0000","last_fullsized":"2024-04-25T01:27:23.405166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6064","reported_seq":6102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322120+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:27:26.322120+0000","last_peered":"2024-04-25T01:27:26.322120+0000","last_clean":"2024-04-25T01:27:26.322120+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:27:26.322120+0000","last_undegraded":"2024-04-25T01:27:26.322120+0000","last_fullsized":"2024-04-25T01:27:26.322120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6062,"num_write_kb":49091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9783","reported_seq":10295,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.349957+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:27:23.349957+0000","last_peered":"2024-04-25T01:27:23.349957+0000","last_clean":"2024-04-25T01:27:23.349957+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:27:23.349957+0000","last_undegraded":"2024-04-25T01:27:23.349957+0000","last_fullsized":"2024-04-25T01:27:23.349957+0000","mapping_epoch":23,"log_start":"27'2453","ondisk_log_start":"27'2453","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2380367,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":482,"num_read_kb":122428,"num_write":7046,"num_write_kb":849139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2499","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.314796+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:27:28.314796+0000","last_peered":"2024-04-25T01:27:28.314796+0000","last_clean":"2024-04-25T01:27:28.314796+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:27:28.314796+0000","last_undegraded":"2024-04-25T01:27:28.314796+0000","last_fullsized":"2024-04-25T01:27:28.314796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2499,"log_dups_size":0,"ondisk_log_size":2499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7885","reported_seq":7923,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:11.111824+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:27:11.111824+0000","last_peered":"2024-04-25T01:27:11.111824+0000","last_clean":"2024-04-25T01:27:11.111824+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:27:11.111824+0000","last_undegraded":"2024-04-25T01:27:11.111824+0000","last_fullsized":"2024-04-25T01:27:11.111824+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7873,"num_write_kb":70555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9266","reported_seq":9744,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.927730+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:27:27.927730+0000","last_peered":"2024-04-25T01:27:27.927730+0000","last_clean":"2024-04-25T01:27:27.927730+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:27:27.927730+0000","last_undegraded":"2024-04-25T01:27:27.927730+0000","last_fullsized":"2024-04-25T01:27:27.927730+0000","mapping_epoch":23,"log_start":"27'1900","ondisk_log_start":"27'1900","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2553648,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":460,"num_read_kb":116840,"num_write":6586,"num_write_kb":921401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2522","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.480449+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:27:23.480449+0000","last_peered":"2024-04-25T01:27:23.480449+0000","last_clean":"2024-04-25T01:27:23.480449+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:27:23.480449+0000","last_undegraded":"2024-04-25T01:27:23.480449+0000","last_fullsized":"2024-04-25T01:27:23.480449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8860","reported_seq":8898,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.115869+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:27:28.115869+0000","last_peered":"2024-04-25T01:27:28.115869+0000","last_clean":"2024-04-25T01:27:28.115869+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:27:28.115869+0000","last_undegraded":"2024-04-25T01:27:28.115869+0000","last_fullsized":"2024-04-25T01:27:28.115869+0000","mapping_epoch":18,"log_start":"27'1500","ondisk_log_start":"27'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7360,"log_dups_size":0,"ondisk_log_size":7360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7300960,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8857,"num_write_kb":62366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9899","reported_seq":10045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.832406+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:27:27.832406+0000","last_peered":"2024-04-25T01:27:27.832406+0000","last_clean":"2024-04-25T01:27:27.832406+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:27:27.832406+0000","last_undegraded":"2024-04-25T01:27:27.832406+0000","last_fullsized":"2024-04-25T01:27:27.832406+0000","mapping_epoch":23,"log_start":"27'2523","ondisk_log_start":"27'2523","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7376,"log_dups_size":0,"ondisk_log_size":7376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071863,"num_objects":22,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":124,"num_read_kb":31496,"num_write":7205,"num_write_kb":913227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.249751+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:27:23.249751+0000","last_peered":"2024-04-25T01:27:23.249751+0000","last_clean":"2024-04-25T01:27:23.249751+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:27:23.249751+0000","last_undegraded":"2024-04-25T01:27:23.249751+0000","last_fullsized":"2024-04-25T01:27:23.249751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7133","reported_seq":8083,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:00.219862+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:27:00.219862+0000","last_peered":"2024-04-25T01:27:00.219862+0000","last_clean":"2024-04-25T01:27:00.219862+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:27:00.219862+0000","last_undegraded":"2024-04-25T01:27:00.219862+0000","last_fullsized":"2024-04-25T01:27:00.219862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7133,"log_dups_size":0,"ondisk_log_size":7133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":912,"num_read_kb":4677,"num_write":7121,"num_write_kb":61966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9048","reported_seq":9471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.972691+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:27:27.972691+0000","last_peered":"2024-04-25T01:27:27.972691+0000","last_clean":"2024-04-25T01:27:27.972691+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:27:27.972691+0000","last_undegraded":"2024-04-25T01:27:27.972691+0000","last_fullsized":"2024-04-25T01:27:27.972691+0000","mapping_epoch":23,"log_start":"27'1700","ondisk_log_start":"27'1700","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7348,"log_dups_size":0,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4129696,"num_objects":18,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":396,"num_read_kb":100584,"num_write":6310,"num_write_kb":903694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2472","reported_seq":2508,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.104533+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:27:23.104533+0000","last_peered":"2024-04-25T01:27:23.104533+0000","last_clean":"2024-04-25T01:27:23.104533+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:27:23.104533+0000","last_undegraded":"2024-04-25T01:27:23.104533+0000","last_fullsized":"2024-04-25T01:27:23.104533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'6806","reported_seq":6844,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.391028+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:27:26.391028+0000","last_peered":"2024-04-25T01:27:26.391028+0000","last_clean":"2024-04-25T01:27:26.391028+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:27:26.391028+0000","last_undegraded":"2024-04-25T01:27:26.391028+0000","last_fullsized":"2024-04-25T01:27:26.391028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6806,"num_write_kb":57773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9170","reported_seq":9528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.406550+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:27:23.406550+0000","last_peered":"2024-04-25T01:27:23.406550+0000","last_clean":"2024-04-25T01:27:23.406550+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:27:23.406550+0000","last_undegraded":"2024-04-25T01:27:23.406550+0000","last_fullsized":"2024-04-25T01:27:23.406550+0000","mapping_epoch":23,"log_start":"27'1826","ondisk_log_start":"27'1826","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3481570,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":329,"num_read_kb":83566,"num_write":6508,"num_write_kb":1104508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2578","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.129101+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:27:23.129101+0000","last_peered":"2024-04-25T01:27:23.129101+0000","last_clean":"2024-04-25T01:27:23.129101+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:27:23.129101+0000","last_undegraded":"2024-04-25T01:27:23.129101+0000","last_fullsized":"2024-04-25T01:27:23.129101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4347","reported_seq":4385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.334344+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:27:26.334344+0000","last_peered":"2024-04-25T01:27:26.334344+0000","last_clean":"2024-04-25T01:27:26.334344+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:27:26.334344+0000","last_undegraded":"2024-04-25T01:27:26.334344+0000","last_fullsized":"2024-04-25T01:27:26.334344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4347,"num_write_kb":39756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:23:30.725659+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:23:30.725659+0000","last_peered":"2024-04-25T01:23:30.725659+0000","last_clean":"2024-04-25T01:23:30.725659+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:23:30.725659+0000","last_undegraded":"2024-04-25T01:23:30.725659+0000","last_fullsized":"2024-04-25T01:23:30.725659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'9521","reported_seq":9782,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.480947+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:27:23.480947+0000","last_peered":"2024-04-25T01:27:23.480947+0000","last_clean":"2024-04-25T01:27:23.480947+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:27:23.480947+0000","last_undegraded":"2024-04-25T01:27:23.480947+0000","last_fullsized":"2024-04-25T01:27:23.480947+0000","mapping_epoch":23,"log_start":"27'1566","ondisk_log_start":"27'1566","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7955,"log_dups_size":0,"ondisk_log_size":7955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1722100,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":58674,"num_write":6884,"num_write_kb":914721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2510","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.556555+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:27:22.556555+0000","last_peered":"2024-04-25T01:27:22.556555+0000","last_clean":"2024-04-25T01:27:22.556555+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:27:22.556555+0000","last_undegraded":"2024-04-25T01:27:22.556555+0000","last_fullsized":"2024-04-25T01:27:22.556555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7379","reported_seq":7416,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:20.151024+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:26:20.151024+0000","last_peered":"2024-04-25T01:26:20.151024+0000","last_clean":"2024-04-25T01:26:20.151024+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:26:20.151024+0000","last_undegraded":"2024-04-25T01:26:20.151024+0000","last_fullsized":"2024-04-25T01:26:20.151024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7379,"log_dups_size":0,"ondisk_log_size":7379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7384,"num_write_kb":58188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9262","reported_seq":9766,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.128091+0000","last_change":"2024-04-25T01:02:13.830001+0000","last_active":"2024-04-25T01:27:23.128091+0000","last_peered":"2024-04-25T01:27:23.128091+0000","last_clean":"2024-04-25T01:27:23.128091+0000","last_became_active":"2024-04-25T01:02:13.829268+0000","last_became_peered":"2024-04-25T01:02:13.829268+0000","last_unstale":"2024-04-25T01:27:23.128091+0000","last_undegraded":"2024-04-25T01:27:23.128091+0000","last_fullsized":"2024-04-25T01:27:23.128091+0000","mapping_epoch":26,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7950,"log_dups_size":0,"ondisk_log_size":7950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:35.237193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1912054,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":468,"num_read_kb":118872,"num_write":6583,"num_write_kb":924937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2548","reported_seq":2583,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.300867+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:27:28.300867+0000","last_peered":"2024-04-25T01:27:28.300867+0000","last_clean":"2024-04-25T01:27:28.300867+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:27:28.300867+0000","last_undegraded":"2024-04-25T01:27:28.300867+0000","last_fullsized":"2024-04-25T01:27:28.300867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7034","reported_seq":7071,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.339472+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:27:26.339472+0000","last_peered":"2024-04-25T01:27:26.339472+0000","last_clean":"2024-04-25T01:27:26.339472+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:27:26.339472+0000","last_undegraded":"2024-04-25T01:27:26.339472+0000","last_fullsized":"2024-04-25T01:27:26.339472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7035,"num_write_kb":58075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9439","reported_seq":10132,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.394196+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:27:23.394196+0000","last_peered":"2024-04-25T01:27:23.394196+0000","last_clean":"2024-04-25T01:27:23.394196+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:27:23.394196+0000","last_undegraded":"2024-04-25T01:27:23.394196+0000","last_fullsized":"2024-04-25T01:27:23.394196+0000","mapping_epoch":23,"log_start":"27'2038","ondisk_log_start":"27'2038","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7401,"log_dups_size":0,"ondisk_log_size":7401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3448832,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":664,"num_read_kb":168656,"num_write":6687,"num_write_kb":894893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2407","reported_seq":2442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.289927+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:27:23.289927+0000","last_peered":"2024-04-25T01:27:23.289927+0000","last_clean":"2024-04-25T01:27:23.289927+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:27:23.289927+0000","last_undegraded":"2024-04-25T01:27:23.289927+0000","last_fullsized":"2024-04-25T01:27:23.289927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2407,"log_dups_size":0,"ondisk_log_size":2407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8734","reported_seq":8771,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:37.183850+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:26:37.183850+0000","last_peered":"2024-04-25T01:26:37.183850+0000","last_clean":"2024-04-25T01:26:37.183850+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:26:37.183850+0000","last_undegraded":"2024-04-25T01:26:37.183850+0000","last_fullsized":"2024-04-25T01:26:37.183850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8734,"log_dups_size":0,"ondisk_log_size":8734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8742,"num_write_kb":79319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9292","reported_seq":10356,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.817906+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:27:27.817906+0000","last_peered":"2024-04-25T01:27:27.817906+0000","last_clean":"2024-04-25T01:27:27.817906+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:27:27.817906+0000","last_undegraded":"2024-04-25T01:27:27.817906+0000","last_fullsized":"2024-04-25T01:27:27.817906+0000","mapping_epoch":23,"log_start":"27'1886","ondisk_log_start":"27'1886","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7406,"log_dups_size":0,"ondisk_log_size":7406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187142,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1035,"num_read_kb":262890,"num_write":6554,"num_write_kb":1048321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.190123+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:27:23.190123+0000","last_peered":"2024-04-25T01:27:23.190123+0000","last_clean":"2024-04-25T01:27:23.190123+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:27:23.190123+0000","last_undegraded":"2024-04-25T01:27:23.190123+0000","last_fullsized":"2024-04-25T01:27:23.190123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5708","reported_seq":5745,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.330594+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:27:26.330594+0000","last_peered":"2024-04-25T01:27:26.330594+0000","last_clean":"2024-04-25T01:27:26.330594+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:27:26.330594+0000","last_undegraded":"2024-04-25T01:27:26.330594+0000","last_fullsized":"2024-04-25T01:27:26.330594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":44693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'8855","reported_seq":9273,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.523452+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:27:27.523452+0000","last_peered":"2024-04-25T01:27:27.523452+0000","last_clean":"2024-04-25T01:27:27.523452+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:27:27.523452+0000","last_undegraded":"2024-04-25T01:27:27.523452+0000","last_fullsized":"2024-04-25T01:27:27.523452+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7245,"log_dups_size":0,"ondisk_log_size":7245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":412999,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":394,"num_read_kb":100076,"num_write":6145,"num_write_kb":854299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2584","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.380863+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:27:23.380863+0000","last_peered":"2024-04-25T01:27:23.380863+0000","last_clean":"2024-04-25T01:27:23.380863+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:27:23.380863+0000","last_undegraded":"2024-04-25T01:27:23.380863+0000","last_fullsized":"2024-04-25T01:27:23.380863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2584,"log_dups_size":0,"ondisk_log_size":2584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'5129","reported_seq":5166,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:15.200879+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:27:15.200879+0000","last_peered":"2024-04-25T01:27:15.200879+0000","last_clean":"2024-04-25T01:27:15.200879+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:27:15.200879+0000","last_undegraded":"2024-04-25T01:27:15.200879+0000","last_fullsized":"2024-04-25T01:27:15.200879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5129,"log_dups_size":0,"ondisk_log_size":5129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":40185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9360","reported_seq":9696,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.479124+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:27:23.479124+0000","last_peered":"2024-04-25T01:27:23.479124+0000","last_clean":"2024-04-25T01:27:23.479124+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:27:23.479124+0000","last_undegraded":"2024-04-25T01:27:23.479124+0000","last_fullsized":"2024-04-25T01:27:23.479124+0000","mapping_epoch":23,"log_start":"27'2023","ondisk_log_start":"27'2023","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6280246,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":308,"num_read_kb":78232,"num_write":6676,"num_write_kb":1180887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2450","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.397998+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:27:23.397998+0000","last_peered":"2024-04-25T01:27:23.397998+0000","last_clean":"2024-04-25T01:27:23.397998+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:27:23.397998+0000","last_undegraded":"2024-04-25T01:27:23.397998+0000","last_fullsized":"2024-04-25T01:27:23.397998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4840","reported_seq":4877,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:14.595711+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:27:14.595711+0000","last_peered":"2024-04-25T01:27:14.595711+0000","last_clean":"2024-04-25T01:27:14.595711+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:27:14.595711+0000","last_undegraded":"2024-04-25T01:27:14.595711+0000","last_fullsized":"2024-04-25T01:27:14.595711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4844,"num_write_kb":44079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9089","reported_seq":9590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.255740+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:27:23.255740+0000","last_peered":"2024-04-25T01:27:23.255740+0000","last_clean":"2024-04-25T01:27:23.255740+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:27:23.255740+0000","last_undegraded":"2024-04-25T01:27:23.255740+0000","last_fullsized":"2024-04-25T01:27:23.255740+0000","mapping_epoch":23,"log_start":"27'2828","ondisk_log_start":"27'2828","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2716911,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":473,"num_read_kb":120142,"num_write":6358,"num_write_kb":909053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2554","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.084782+0000","last_change":"2024-04-25T01:02:13.744191+0000","last_active":"2024-04-25T01:27:23.084782+0000","last_peered":"2024-04-25T01:27:23.084782+0000","last_clean":"2024-04-25T01:27:23.084782+0000","last_became_active":"2024-04-25T01:02:13.743832+0000","last_became_peered":"2024-04-25T01:02:13.743832+0000","last_unstale":"2024-04-25T01:27:23.084782+0000","last_undegraded":"2024-04-25T01:27:23.084782+0000","last_fullsized":"2024-04-25T01:27:23.084782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"27'2887","reported_seq":2924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.345321+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:27:26.345321+0000","last_peered":"2024-04-25T01:27:26.345321+0000","last_clean":"2024-04-25T01:27:26.345321+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:27:26.345321+0000","last_undegraded":"2024-04-25T01:27:26.345321+0000","last_fullsized":"2024-04-25T01:27:26.345321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2887,"log_dups_size":0,"ondisk_log_size":2887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2884,"num_write_kb":30449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9218","reported_seq":9460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.036523+0000","last_change":"2024-04-25T01:02:13.830353+0000","last_active":"2024-04-25T01:27:23.036523+0000","last_peered":"2024-04-25T01:27:23.036523+0000","last_clean":"2024-04-25T01:27:23.036523+0000","last_became_active":"2024-04-25T01:02:13.829847+0000","last_became_peered":"2024-04-25T01:02:13.829847+0000","last_unstale":"2024-04-25T01:27:23.036523+0000","last_undegraded":"2024-04-25T01:27:23.036523+0000","last_fullsized":"2024-04-25T01:27:23.036523+0000","mapping_epoch":26,"log_start":"27'2967","ondisk_log_start":"27'2967","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:00.933599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1903266,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":52578,"num_write":6608,"num_write_kb":922819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2517","reported_seq":2552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.153861+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:27:23.153861+0000","last_peered":"2024-04-25T01:27:23.153861+0000","last_clean":"2024-04-25T01:27:23.153861+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:27:23.153861+0000","last_undegraded":"2024-04-25T01:27:23.153861+0000","last_fullsized":"2024-04-25T01:27:23.153861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2517,"log_dups_size":0,"ondisk_log_size":2517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6766","reported_seq":6803,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:03.691415+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:27:03.691415+0000","last_peered":"2024-04-25T01:27:03.691415+0000","last_clean":"2024-04-25T01:27:03.691415+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:27:03.691415+0000","last_undegraded":"2024-04-25T01:27:03.691415+0000","last_fullsized":"2024-04-25T01:27:03.691415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6766,"log_dups_size":0,"ondisk_log_size":6766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6774,"num_write_kb":57696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9611","reported_seq":9984,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.485448+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:27:28.485448+0000","last_peered":"2024-04-25T01:27:28.485448+0000","last_clean":"2024-04-25T01:27:28.485448+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:27:28.485448+0000","last_undegraded":"2024-04-25T01:27:28.485448+0000","last_fullsized":"2024-04-25T01:27:28.485448+0000","mapping_epoch":23,"log_start":"27'2214","ondisk_log_start":"27'2214","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2536524,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":349,"num_read_kb":88646,"num_write":6979,"num_write_kb":1079633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2454","reported_seq":2488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.321625+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:27:23.321625+0000","last_peered":"2024-04-25T01:27:23.321625+0000","last_clean":"2024-04-25T01:27:23.321625+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:27:23.321625+0000","last_undegraded":"2024-04-25T01:27:23.321625+0000","last_fullsized":"2024-04-25T01:27:23.321625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2454,"log_dups_size":0,"ondisk_log_size":2454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10280","reported_seq":10909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:15.207946+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:27:15.207946+0000","last_peered":"2024-04-25T01:27:15.207946+0000","last_clean":"2024-04-25T01:27:15.207946+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:27:15.207946+0000","last_undegraded":"2024-04-25T01:27:15.207946+0000","last_fullsized":"2024-04-25T01:27:15.207946+0000","mapping_epoch":18,"log_start":"27'2300","ondisk_log_start":"27'2300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":594,"num_read_kb":4482,"num_write":10274,"num_write_kb":76426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9142","reported_seq":9483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.928290+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:27:27.928290+0000","last_peered":"2024-04-25T01:27:27.928290+0000","last_clean":"2024-04-25T01:27:27.928290+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:27:27.928290+0000","last_undegraded":"2024-04-25T01:27:27.928290+0000","last_fullsized":"2024-04-25T01:27:27.928290+0000","mapping_epoch":23,"log_start":"27'1734","ondisk_log_start":"27'1734","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7408,"log_dups_size":0,"ondisk_log_size":7408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1592989,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":312,"num_read_kb":79248,"num_write":6479,"num_write_kb":938399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2545","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.035246+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:27:23.035246+0000","last_peered":"2024-04-25T01:27:23.035246+0000","last_clean":"2024-04-25T01:27:23.035246+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:27:23.035246+0000","last_undegraded":"2024-04-25T01:27:23.035246+0000","last_fullsized":"2024-04-25T01:27:23.035246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2545,"log_dups_size":0,"ondisk_log_size":2545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6456","reported_seq":6492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.396435+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:27:26.396435+0000","last_peered":"2024-04-25T01:27:26.396435+0000","last_clean":"2024-04-25T01:27:26.396435+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:27:26.396435+0000","last_undegraded":"2024-04-25T01:27:26.396435+0000","last_fullsized":"2024-04-25T01:27:26.396435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6456,"log_dups_size":0,"ondisk_log_size":6456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6471,"num_write_kb":41578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'8950","reported_seq":9484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:27.661822+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:27:27.661822+0000","last_peered":"2024-04-25T01:27:27.661822+0000","last_clean":"2024-04-25T01:27:27.661822+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:27:27.661822+0000","last_undegraded":"2024-04-25T01:27:27.661822+0000","last_fullsized":"2024-04-25T01:27:27.661822+0000","mapping_epoch":23,"log_start":"27'1803","ondisk_log_start":"27'1803","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7147,"log_dups_size":0,"ondisk_log_size":7147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2560955,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":508,"num_read_kb":129032,"num_write":6252,"num_write_kb":1000416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2560","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.148657+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:27:23.148657+0000","last_peered":"2024-04-25T01:27:23.148657+0000","last_clean":"2024-04-25T01:27:23.148657+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:27:23.148657+0000","last_undegraded":"2024-04-25T01:27:23.148657+0000","last_fullsized":"2024-04-25T01:27:23.148657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2560,"log_dups_size":0,"ondisk_log_size":2560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6614","reported_seq":6650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.363784+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:27:26.363784+0000","last_peered":"2024-04-25T01:27:26.363784+0000","last_clean":"2024-04-25T01:27:26.363784+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:27:26.363784+0000","last_undegraded":"2024-04-25T01:27:26.363784+0000","last_fullsized":"2024-04-25T01:27:26.363784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6614,"log_dups_size":0,"ondisk_log_size":6614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6608,"num_write_kb":53556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9193","reported_seq":9400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.462589+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:27:23.462589+0000","last_peered":"2024-04-25T01:27:23.462589+0000","last_clean":"2024-04-25T01:27:23.462589+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:27:23.462589+0000","last_undegraded":"2024-04-25T01:27:23.462589+0000","last_fullsized":"2024-04-25T01:27:23.462589+0000","mapping_epoch":23,"log_start":"27'2910","ondisk_log_start":"27'2910","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:22.580754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1149248,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6520,"num_write_kb":962512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2543","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.312816+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:27:28.312816+0000","last_peered":"2024-04-25T01:27:28.312816+0000","last_clean":"2024-04-25T01:27:28.312816+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:27:28.312816+0000","last_undegraded":"2024-04-25T01:27:28.312816+0000","last_fullsized":"2024-04-25T01:27:28.312816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2543,"log_dups_size":0,"ondisk_log_size":2543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6620","reported_seq":7475,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:25:50.038282+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:25:50.038282+0000","last_peered":"2024-04-25T01:25:50.038282+0000","last_clean":"2024-04-25T01:25:50.038282+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:25:50.038282+0000","last_undegraded":"2024-04-25T01:25:50.038282+0000","last_fullsized":"2024-04-25T01:25:50.038282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6620,"log_dups_size":0,"ondisk_log_size":6620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":821,"num_read_kb":4618,"num_write":6622,"num_write_kb":49469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9979","reported_seq":10043,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.444337+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:27:23.444337+0000","last_peered":"2024-04-25T01:27:23.444337+0000","last_clean":"2024-04-25T01:27:23.444337+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:27:23.444337+0000","last_undegraded":"2024-04-25T01:27:23.444337+0000","last_fullsized":"2024-04-25T01:27:23.444337+0000","mapping_epoch":23,"log_start":"27'3717","ondisk_log_start":"27'3717","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6262,"log_dups_size":0,"ondisk_log_size":6262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1707840,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":35,"num_read_kb":8890,"num_write":7259,"num_write_kb":1063948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9051","reported_seq":9308,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.341237+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:27:23.341237+0000","last_peered":"2024-04-25T01:27:23.341237+0000","last_clean":"2024-04-25T01:27:23.341237+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:27:23.341237+0000","last_undegraded":"2024-04-25T01:27:23.341237+0000","last_fullsized":"2024-04-25T01:27:23.341237+0000","mapping_epoch":23,"log_start":"27'1719","ondisk_log_start":"27'1719","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":227,"num_read_kb":57658,"num_write":6452,"num_write_kb":909285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2475","reported_seq":2509,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.998206+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:27:22.998206+0000","last_peered":"2024-04-25T01:27:22.998206+0000","last_clean":"2024-04-25T01:27:22.998206+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:27:22.998206+0000","last_undegraded":"2024-04-25T01:27:22.998206+0000","last_fullsized":"2024-04-25T01:27:22.998206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9899","reported_seq":9935,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322490+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:27:26.322490+0000","last_peered":"2024-04-25T01:27:26.322490+0000","last_clean":"2024-04-25T01:27:26.322490+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:27:26.322490+0000","last_undegraded":"2024-04-25T01:27:26.322490+0000","last_fullsized":"2024-04-25T01:27:26.322490+0000","mapping_epoch":18,"log_start":"27'2500","ondisk_log_start":"27'2500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":68120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8927","reported_seq":9786,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.374438+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:27:23.374438+0000","last_peered":"2024-04-25T01:27:23.374438+0000","last_clean":"2024-04-25T01:27:23.374438+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:27:23.374438+0000","last_undegraded":"2024-04-25T01:27:23.374438+0000","last_fullsized":"2024-04-25T01:27:23.374438+0000","mapping_epoch":23,"log_start":"27'1000","ondisk_log_start":"27'1000","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7927,"log_dups_size":0,"ondisk_log_size":7927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2619556,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":829,"num_read_kb":210566,"num_write":6293,"num_write_kb":892044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2577","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.071456+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:27:23.071456+0000","last_peered":"2024-04-25T01:27:23.071456+0000","last_clean":"2024-04-25T01:27:23.071456+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:27:23.071456+0000","last_undegraded":"2024-04-25T01:27:23.071456+0000","last_fullsized":"2024-04-25T01:27:23.071456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5341","reported_seq":5377,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:13.925537+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:26:13.925537+0000","last_peered":"2024-04-25T01:26:13.925537+0000","last_clean":"2024-04-25T01:26:13.925537+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:26:13.925537+0000","last_undegraded":"2024-04-25T01:26:13.925537+0000","last_fullsized":"2024-04-25T01:26:13.925537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5341,"num_write_kb":44512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9496","reported_seq":9532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:26:14.123973+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:26:14.123973+0000","last_peered":"2024-04-25T01:26:14.123973+0000","last_clean":"2024-04-25T01:26:14.123973+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:26:14.123973+0000","last_undegraded":"2024-04-25T01:26:14.123973+0000","last_fullsized":"2024-04-25T01:26:14.123973+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":9507,"num_write_kb":71968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2515","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.271703+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:27:23.271703+0000","last_peered":"2024-04-25T01:27:23.271703+0000","last_clean":"2024-04-25T01:27:23.271703+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:27:23.271703+0000","last_undegraded":"2024-04-25T01:27:23.271703+0000","last_fullsized":"2024-04-25T01:27:23.271703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9444","reported_seq":9599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.419393+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:27:23.419393+0000","last_peered":"2024-04-25T01:27:23.419393+0000","last_clean":"2024-04-25T01:27:23.419393+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:27:23.419393+0000","last_undegraded":"2024-04-25T01:27:23.419393+0000","last_fullsized":"2024-04-25T01:27:23.419393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4296704,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":127,"num_read_kb":32258,"num_write":6733,"num_write_kb":1000281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8023","reported_seq":8059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:24:51.994967+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:24:51.994967+0000","last_peered":"2024-04-25T01:24:51.994967+0000","last_clean":"2024-04-25T01:24:51.994967+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:24:51.994967+0000","last_undegraded":"2024-04-25T01:24:51.994967+0000","last_fullsized":"2024-04-25T01:24:51.994967+0000","mapping_epoch":18,"log_start":"27'1700","ondisk_log_start":"27'1700","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8018,"num_write_kb":66584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2558","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.048774+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:27:23.048774+0000","last_peered":"2024-04-25T01:27:23.048774+0000","last_clean":"2024-04-25T01:27:23.048774+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:27:23.048774+0000","last_undegraded":"2024-04-25T01:27:23.048774+0000","last_fullsized":"2024-04-25T01:27:23.048774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9429","reported_seq":9825,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.252529+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:27:23.252529+0000","last_peered":"2024-04-25T01:27:23.252529+0000","last_clean":"2024-04-25T01:27:23.252529+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:27:23.252529+0000","last_undegraded":"2024-04-25T01:27:23.252529+0000","last_fullsized":"2024-04-25T01:27:23.252529+0000","mapping_epoch":23,"log_start":"27'3125","ondisk_log_start":"27'3125","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6304,"log_dups_size":0,"ondisk_log_size":6304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:06.642753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6485238,"num_objects":15,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":368,"num_read_kb":93472,"num_write":6715,"num_write_kb":1049905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2448","reported_seq":2484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.142813+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:27:23.142813+0000","last_peered":"2024-04-25T01:27:23.142813+0000","last_clean":"2024-04-25T01:27:23.142813+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:27:23.142813+0000","last_undegraded":"2024-04-25T01:27:23.142813+0000","last_fullsized":"2024-04-25T01:27:23.142813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2448,"log_dups_size":0,"ondisk_log_size":2448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2578","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.052048+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:27:23.052048+0000","last_peered":"2024-04-25T01:27:23.052048+0000","last_clean":"2024-04-25T01:27:23.052048+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:27:23.052048+0000","last_undegraded":"2024-04-25T01:27:23.052048+0000","last_fullsized":"2024-04-25T01:27:23.052048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2493","reported_seq":2529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.401417+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:27:28.401417+0000","last_peered":"2024-04-25T01:27:28.401417+0000","last_clean":"2024-04-25T01:27:28.401417+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:27:28.401417+0000","last_undegraded":"2024-04-25T01:27:28.401417+0000","last_fullsized":"2024-04-25T01:27:28.401417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2562","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.122836+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:27:23.122836+0000","last_peered":"2024-04-25T01:27:23.122836+0000","last_clean":"2024-04-25T01:27:23.122836+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:27:23.122836+0000","last_undegraded":"2024-04-25T01:27:23.122836+0000","last_fullsized":"2024-04-25T01:27:23.122836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.340997+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:27:23.340997+0000","last_peered":"2024-04-25T01:27:23.340997+0000","last_clean":"2024-04-25T01:27:23.340997+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:27:23.340997+0000","last_undegraded":"2024-04-25T01:27:23.340997+0000","last_fullsized":"2024-04-25T01:27:23.340997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2445","reported_seq":2480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.369622+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:27:23.369622+0000","last_peered":"2024-04-25T01:27:23.369622+0000","last_clean":"2024-04-25T01:27:23.369622+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:27:23.369622+0000","last_undegraded":"2024-04-25T01:27:23.369622+0000","last_fullsized":"2024-04-25T01:27:23.369622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2573","reported_seq":2608,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.402625+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:27:23.402625+0000","last_peered":"2024-04-25T01:27:23.402625+0000","last_clean":"2024-04-25T01:27:23.402625+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:27:23.402625+0000","last_undegraded":"2024-04-25T01:27:23.402625+0000","last_fullsized":"2024-04-25T01:27:23.402625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2538","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.170149+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:27:23.170149+0000","last_peered":"2024-04-25T01:27:23.170149+0000","last_clean":"2024-04-25T01:27:23.170149+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:27:23.170149+0000","last_undegraded":"2024-04-25T01:27:23.170149+0000","last_fullsized":"2024-04-25T01:27:23.170149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2538,"log_dups_size":0,"ondisk_log_size":2538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2529","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.289774+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:27:28.289774+0000","last_peered":"2024-04-25T01:27:28.289774+0000","last_clean":"2024-04-25T01:27:28.289774+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:27:28.289774+0000","last_undegraded":"2024-04-25T01:27:28.289774+0000","last_fullsized":"2024-04-25T01:27:28.289774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2500","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.365002+0000","last_change":"2024-04-25T01:01:33.405268+0000","last_active":"2024-04-25T01:27:23.365002+0000","last_peered":"2024-04-25T01:27:23.365002+0000","last_clean":"2024-04-25T01:27:23.365002+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:27:23.365002+0000","last_undegraded":"2024-04-25T01:27:23.365002+0000","last_fullsized":"2024-04-25T01:27:23.365002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2557","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.475740+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:27:28.475740+0000","last_peered":"2024-04-25T01:27:28.475740+0000","last_clean":"2024-04-25T01:27:28.475740+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:27:28.475740+0000","last_undegraded":"2024-04-25T01:27:28.475740+0000","last_fullsized":"2024-04-25T01:27:28.475740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2567","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.273548+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:27:23.273548+0000","last_peered":"2024-04-25T01:27:23.273548+0000","last_clean":"2024-04-25T01:27:23.273548+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:27:23.273548+0000","last_undegraded":"2024-04-25T01:27:23.273548+0000","last_fullsized":"2024-04-25T01:27:23.273548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2572","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.152624+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:27:23.152624+0000","last_peered":"2024-04-25T01:27:23.152624+0000","last_clean":"2024-04-25T01:27:23.152624+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:27:23.152624+0000","last_undegraded":"2024-04-25T01:27:23.152624+0000","last_fullsized":"2024-04-25T01:27:23.152624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2572,"log_dups_size":0,"ondisk_log_size":2572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2573","reported_seq":2608,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.393729+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:27:23.393729+0000","last_peered":"2024-04-25T01:27:23.393729+0000","last_clean":"2024-04-25T01:27:23.393729+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:27:23.393729+0000","last_undegraded":"2024-04-25T01:27:23.393729+0000","last_fullsized":"2024-04-25T01:27:23.393729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2512","reported_seq":2547,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.365445+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:27:23.365445+0000","last_peered":"2024-04-25T01:27:23.365445+0000","last_clean":"2024-04-25T01:27:23.365445+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:27:23.365445+0000","last_undegraded":"2024-04-25T01:27:23.365445+0000","last_fullsized":"2024-04-25T01:27:23.365445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2512,"log_dups_size":0,"ondisk_log_size":2512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2588","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.349289+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:27:28.349289+0000","last_peered":"2024-04-25T01:27:28.349289+0000","last_clean":"2024-04-25T01:27:28.349289+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:27:28.349289+0000","last_undegraded":"2024-04-25T01:27:28.349289+0000","last_fullsized":"2024-04-25T01:27:28.349289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2509","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.305277+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:27:28.305277+0000","last_peered":"2024-04-25T01:27:28.305277+0000","last_clean":"2024-04-25T01:27:28.305277+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:27:28.305277+0000","last_undegraded":"2024-04-25T01:27:28.305277+0000","last_fullsized":"2024-04-25T01:27:28.305277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2509,"log_dups_size":0,"ondisk_log_size":2509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2625","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.045603+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:27:23.045603+0000","last_peered":"2024-04-25T01:27:23.045603+0000","last_clean":"2024-04-25T01:27:23.045603+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:27:23.045603+0000","last_undegraded":"2024-04-25T01:27:23.045603+0000","last_fullsized":"2024-04-25T01:27:23.045603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2482","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.837016+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:27:22.837016+0000","last_peered":"2024-04-25T01:27:22.837016+0000","last_clean":"2024-04-25T01:27:22.837016+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:27:22.837016+0000","last_undegraded":"2024-04-25T01:27:22.837016+0000","last_fullsized":"2024-04-25T01:27:22.837016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2471","reported_seq":2505,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.182821+0000","last_change":"2024-04-25T01:01:33.404972+0000","last_active":"2024-04-25T01:27:23.182821+0000","last_peered":"2024-04-25T01:27:23.182821+0000","last_clean":"2024-04-25T01:27:23.182821+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:27:23.182821+0000","last_undegraded":"2024-04-25T01:27:23.182821+0000","last_fullsized":"2024-04-25T01:27:23.182821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2431","reported_seq":2465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.795020+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:27:22.795020+0000","last_peered":"2024-04-25T01:27:22.795020+0000","last_clean":"2024-04-25T01:27:22.795020+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:27:22.795020+0000","last_undegraded":"2024-04-25T01:27:22.795020+0000","last_fullsized":"2024-04-25T01:27:22.795020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2431,"log_dups_size":0,"ondisk_log_size":2431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2464","reported_seq":2503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.288941+0000","last_change":"2024-04-25T01:02:13.749183+0000","last_active":"2024-04-25T01:27:23.288941+0000","last_peered":"2024-04-25T01:27:23.288941+0000","last_clean":"2024-04-25T01:27:23.288941+0000","last_became_active":"2024-04-25T01:02:13.748901+0000","last_became_peered":"2024-04-25T01:02:13.748901+0000","last_unstale":"2024-04-25T01:27:23.288941+0000","last_undegraded":"2024-04-25T01:27:23.288941+0000","last_fullsized":"2024-04-25T01:27:23.288941+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2464,"log_dups_size":0,"ondisk_log_size":2464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:54.284115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2596","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.443677+0000","last_change":"2024-04-25T01:02:13.750135+0000","last_active":"2024-04-25T01:27:23.443677+0000","last_peered":"2024-04-25T01:27:23.443677+0000","last_clean":"2024-04-25T01:27:23.443677+0000","last_became_active":"2024-04-25T01:02:13.749859+0000","last_became_peered":"2024-04-25T01:02:13.749859+0000","last_unstale":"2024-04-25T01:27:23.443677+0000","last_undegraded":"2024-04-25T01:27:23.443677+0000","last_fullsized":"2024-04-25T01:27:23.443677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:18.550102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2552","reported_seq":2586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.351165+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:27:23.351165+0000","last_peered":"2024-04-25T01:27:23.351165+0000","last_clean":"2024-04-25T01:27:23.351165+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:27:23.351165+0000","last_undegraded":"2024-04-25T01:27:23.351165+0000","last_fullsized":"2024-04-25T01:27:23.351165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2552,"log_dups_size":0,"ondisk_log_size":2552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2498","reported_seq":2518,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.384142+0000","last_change":"2024-04-25T01:02:13.828931+0000","last_active":"2024-04-25T01:27:23.384142+0000","last_peered":"2024-04-25T01:27:23.384142+0000","last_clean":"2024-04-25T01:27:23.384142+0000","last_became_active":"2024-04-25T01:02:13.828599+0000","last_became_peered":"2024-04-25T01:02:13.828599+0000","last_unstale":"2024-04-25T01:27:23.384142+0000","last_undegraded":"2024-04-25T01:27:23.384142+0000","last_fullsized":"2024-04-25T01:27:23.384142+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:20.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.39","version":"27'2514","reported_seq":2548,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.084091+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:27:23.084091+0000","last_peered":"2024-04-25T01:27:23.084091+0000","last_clean":"2024-04-25T01:27:23.084091+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:27:23.084091+0000","last_undegraded":"2024-04-25T01:27:23.084091+0000","last_fullsized":"2024-04-25T01:27:23.084091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2468","reported_seq":2502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.359740+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:27:28.359740+0000","last_peered":"2024-04-25T01:27:28.359740+0000","last_clean":"2024-04-25T01:27:28.359740+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:27:28.359740+0000","last_undegraded":"2024-04-25T01:27:28.359740+0000","last_fullsized":"2024-04-25T01:27:28.359740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2468,"log_dups_size":0,"ondisk_log_size":2468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2492","reported_seq":2526,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:22.629304+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:27:22.629304+0000","last_peered":"2024-04-25T01:27:22.629304+0000","last_clean":"2024-04-25T01:27:22.629304+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:27:22.629304+0000","last_undegraded":"2024-04-25T01:27:22.629304+0000","last_fullsized":"2024-04-25T01:27:22.629304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2492,"log_dups_size":0,"ondisk_log_size":2492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2557","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.305843+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:27:28.305843+0000","last_peered":"2024-04-25T01:27:28.305843+0000","last_clean":"2024-04-25T01:27:28.305843+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:27:28.305843+0000","last_undegraded":"2024-04-25T01:27:28.305843+0000","last_fullsized":"2024-04-25T01:27:28.305843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2606","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.217124+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:27:23.217124+0000","last_peered":"2024-04-25T01:27:23.217124+0000","last_clean":"2024-04-25T01:27:23.217124+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:27:23.217124+0000","last_undegraded":"2024-04-25T01:27:23.217124+0000","last_fullsized":"2024-04-25T01:27:23.217124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2458","reported_seq":2494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.190500+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:27:23.190500+0000","last_peered":"2024-04-25T01:27:23.190500+0000","last_clean":"2024-04-25T01:27:23.190500+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:27:23.190500+0000","last_undegraded":"2024-04-25T01:27:23.190500+0000","last_fullsized":"2024-04-25T01:27:23.190500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2458,"log_dups_size":0,"ondisk_log_size":2458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2434","reported_seq":2470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:23.399721+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:27:23.399721+0000","last_peered":"2024-04-25T01:27:23.399721+0000","last_clean":"2024-04-25T01:27:23.399721+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:27:23.399721+0000","last_undegraded":"2024-04-25T01:27:23.399721+0000","last_fullsized":"2024-04-25T01:27:23.399721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":78633850,"num_objects":334,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":12020,"num_read_kb":3053080,"num_write":211822,"num_write_kb":30669379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160497664,"data_stored":160497664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232144,"ondisk_log_size":232144,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161709,"ondisk_log_size":161709,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":230335553,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":3110,"num_read_kb":19066,"num_write":209236,"num_write_kb":1676068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":70,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459538432,"data_stored":459478144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194116,"ondisk_log_size":194116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575165,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":956092,"kb_used_data":58612,"kb_used_omap":1,"kb_used_meta":897470,"kb_avail":93415748,"statfs":{"total":96636764160,"available":95657725952,"internally_reserved":0,"allocated":60018688,"data_stored":59837569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":919009738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575161,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1202892,"kb_used_data":89156,"kb_used_omap":1,"kb_used_meta":1113726,"kb_avail":93168948,"statfs":{"total":96636764160,"available":95405002752,"internally_reserved":0,"allocated":91295744,"data_stored":91116082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1140455878},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1009360,"kb_used_data":57864,"kb_used_omap":1,"kb_used_meta":951486,"kb_avail":93362480,"statfs":{"total":96636764160,"available":95603179520,"internally_reserved":0,"allocated":59252736,"data_stored":59074738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":974322117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575162,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1013728,"kb_used_data":80280,"kb_used_omap":1,"kb_used_meta":933438,"kb_avail":93358112,"statfs":{"total":96636764160,"available":95598706688,"internally_reserved":0,"allocated":82206720,"data_stored":82022774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":955840965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1154908,"kb_used_data":99284,"kb_used_omap":1,"kb_used_meta":1055614,"kb_avail":93216932,"statfs":{"total":96636764160,"available":95454138368,"internally_reserved":0,"allocated":101666816,"data_stored":101487512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1080949191},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694676,"kb_used_data":72140,"kb_used_omap":1,"kb_used_meta":622526,"kb_avail":93677164,"statfs":{"total":96636764160,"available":95925415936,"internally_reserved":0,"allocated":73871360,"data_stored":73699576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637467077},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1251780,"kb_used_data":79612,"kb_used_omap":1,"kb_used_meta":1172158,"kb_avail":93120060,"statfs":{"total":96636764160,"available":95354941440,"internally_reserved":0,"allocated":81522688,"data_stored":81341823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1200290245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":850464,"kb_used_data":71128,"kb_used_omap":1,"kb_used_meta":779326,"kb_avail":93521376,"statfs":{"total":96636764160,"available":95765889024,"internally_reserved":0,"allocated":72835072,"data_stored":72648086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":798030277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54538240,"data_stored":54526590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60657664,"data_stored":60648583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54525952,"data_stored":54525952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40861696,"data_stored":40855482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":67837952,"data_stored":67830074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":46878720,"data_stored":46872937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17412096,"data_stored":17412096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26370048,"data_stored":26370048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19054592,"data_stored":19054592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27361280,"data_stored":27361280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17469440,"data_stored":17469440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":17408000,"data_stored":17408000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23166976,"data_stored":23166976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12255232,"data_stored":12255232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:27:30.055 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:27:30.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.209+0000 7f1903dce640 1 Processor -- start 2024-04-25T01:27:30.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.209+0000 7f1903dce640 1 -- start start 2024-04-25T01:27:30.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f1903dce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f1903dce640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18fc153da0 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f1903dce640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18fc0c8350 con 0x7f18fc0c4e50 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f1903dce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f18fc0c6990 con 0x7f18fc070e10 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f190236b640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f18fc0c6eb0 0x7f18fc15da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60100/0 (socket says 172.21.15.129:60100) 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f190236b640 1 -- 172.21.15.129:0/2291808763 learned_addr learned my addr 172.21.15.129:0/2291808763 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.210+0000 7f1901b6a640 1 --2- 172.21.15.129:0/2291808763 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1900b68640 1 -- 172.21.15.129:0/2291808763 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2339005161 0 0) 0x7f18fc153da0 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1900b68640 1 -- 172.21.15.129:0/2291808763 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18e8003520 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1900b68640 1 -- 172.21.15.129:0/2291808763 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2076325203 0 0) 0x7f18fc0c8350 con 0x7f18fc0c4e50 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1901b6a640 1 -- 172.21.15.129:0/2291808763 >> v1:172.21.15.171:6789/0 conn(0x7f18fc0c4e50 legacy=0x7f18fc0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1901b6a640 1 -- 172.21.15.129:0/2291808763 >> v1:172.21.15.129:6789/0 conn(0x7f18fc0c6eb0 legacy=0x7f18fc15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1901b6a640 1 -- 172.21.15.129:0/2291808763 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc153ab0 con 0x7f18fc070e10 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.211+0000 7f1901b6a640 1 --2- 172.21.15.129:0/2291808763 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc06e8c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0097d0 tx=0x7f18ec02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6785cb9d63e02676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1900b68640 1 -- 172.21.15.129:0/2291808763 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec037070 con 0x7f18fc070e10 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1900b68640 1 -- 172.21.15.129:0/2291808763 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ec032440 con 0x7f18fc070e10 2024-04-25T01:27:30.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1900b68640 1 -- 172.21.15.129:0/2291808763 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec032810 con 0x7f18fc070e10 2024-04-25T01:27:30.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 -- 172.21.15.129:0/2291808763 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 msgr2=0x7f18fc06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 --2- 172.21.15.129:0/2291808763 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc06e8c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0097d0 tx=0x7f18ec02a210 comp rx=0 tx=0).stop 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 -- 172.21.15.129:0/2291808763 shutdown_connections 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 --2- 172.21.15.129:0/2291808763 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc06e8c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 -- 172.21.15.129:0/2291808763 >> 172.21.15.129:0/2291808763 conn(0x7f18fc0c5490 msgr2=0x7f18fc0c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 -- 172.21.15.129:0/2291808763 shutdown_connections 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.212+0000 7f1903dce640 1 -- 172.21.15.129:0/2291808763 wait complete. 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.213+0000 7f1903dce640 1 Processor -- start 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.213+0000 7f1903dce640 1 -- start start 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1903dce640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1903dce640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18fc0c4e50 0x7f18fc1e9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1903dce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 0x7f18fc0a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1903dce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc0aade0 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1903dce640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc0c8630 con 0x7f18fc0c4e50 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1903dce640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f18fc0aaf60 con 0x7f18fc070e10 2024-04-25T01:27:30.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 0x7f18fc0a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 0x7f18fc0a9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55344/0 (socket says 172.21.15.129:55344) 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 -- 172.21.15.129:0/424216143 learned_addr learned my addr 172.21.15.129:0/424216143 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1901b6a640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1901369640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18fc0c4e50 0x7f18fc1e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 -- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 msgr2=0x7f18fc154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 -- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18fc0c4e50 msgr2=0x7f18fc1e9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18fc0c4e50 0x7f18fc1e9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f190236b640 1 -- 172.21.15.129:0/424216143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc157540 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.214+0000 7f1901b6a640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.215+0000 7f190236b640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 0x7f18fc0a9970 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cb40 tx=0x7f18f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.215+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4017070 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.215+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f400aca0 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.215+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4012620 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.215+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc1e96a0 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.215+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc0ab9b0 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.216+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18fc075550 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.216+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18f401a070 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.217+0000 7f18f27fc640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f18e006be30 0x7f18e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.217+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f18f408af60 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.217+0000 7f1901b6a640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f18e006be30 0x7f18e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.217+0000 7f1901b6a640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f18e006be30 0x7f18e006e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18ec034000 tx=0x7f18ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:30.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.221+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18f4058f50 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.404+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18fc0c7f50 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.405+0000 7f18f27fc640 1 -- 172.21.15.129:0/424216143 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18891 (secure 0 0 0) 0x7f18f405d250 con 0x7f18fc0c6eb0 2024-04-25T01:27:30.385 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:30.385 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:02:13.734125+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:27:30.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f18e006be30 msgr2=0x7f18e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.387 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f18e006be30 0x7f18e006e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18ec034000 tx=0x7f18ec02e000 comp rx=0 tx=0).stop 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 msgr2=0x7f18fc0a9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 0x7f18fc0a9970 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cb40 tx=0x7f18f400a060 comp rx=0 tx=0).stop 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 shutdown_connections 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f18e006be30 0x7f18e006e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18fc0c6eb0 0x7f18fc0a9970 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18fc0c4e50 0x7f18fc1e9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 --2- 172.21.15.129:0/424216143 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f18fc070e10 0x7f18fc154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 >> 172.21.15.129:0/424216143 conn(0x7f18fc0c5490 msgr2=0x7f18fc153180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 shutdown_connections 2024-04-25T01:27:30.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:30.407+0000 7f1903dce640 1 -- 172.21.15.129:0/424216143 wait complete. 2024-04-25T01:27:31.399 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T01:27:31.399 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:31.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.522+0000 7ff76aa72640 1 Processor -- start 2024-04-25T01:27:31.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.522+0000 7ff76aa72640 1 -- start start 2024-04-25T01:27:31.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76aa72640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7640c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76aa72640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76406cfe0 con 0x7ff7640c8b20 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76aa72640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff764155490 con 0x7ff7640c9030 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76aa72640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff764155dc0 con 0x7ff764152bb0 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76900f640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7ff7640c8b20 0x7ff76415fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60114/0 (socket says 172.21.15.129:60114) 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76900f640 1 -- 172.21.15.129:0/2405850880 learned_addr learned my addr 172.21.15.129:0/2405850880 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76880e640 1 --2- 172.21.15.129:0/2405850880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7640c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff7637fe640 1 -- 172.21.15.129:0/2405850880 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034885996 0 0) 0x7ff76406cfe0 con 0x7ff7640c8b20 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff7637fe640 1 -- 172.21.15.129:0/2405850880 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff754003520 con 0x7ff7640c8b20 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff7637fe640 1 -- 172.21.15.129:0/2405850880 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092174778 0 0) 0x7ff764155490 con 0x7ff7640c9030 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76880e640 1 -- 172.21.15.129:0/2405850880 >> v1:172.21.15.171:6789/0 conn(0x7ff7640c9030 legacy=0x7ff764158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76880e640 1 -- 172.21.15.129:0/2405850880 >> v1:172.21.15.129:6789/0 conn(0x7ff7640c8b20 legacy=0x7ff76415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.523+0000 7ff76880e640 1 -- 172.21.15.129:0/2405850880 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff764155770 con 0x7ff764152bb0 2024-04-25T01:27:31.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.524+0000 7ff76880e640 1 --2- 172.21.15.129:0/2405850880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7640c55a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7580097d0 tx=0x7ff75802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75265b6dfde17835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.524+0000 7ff7637fe640 1 -- 172.21.15.129:0/2405850880 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758037070 con 0x7ff764152bb0 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.524+0000 7ff7637fe640 1 -- 172.21.15.129:0/2405850880 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff758032440 con 0x7ff764152bb0 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.524+0000 7ff7637fe640 1 -- 172.21.15.129:0/2405850880 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758032810 con 0x7ff764152bb0 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 -- 172.21.15.129:0/2405850880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 msgr2=0x7ff7640c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 --2- 172.21.15.129:0/2405850880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7640c55a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7580097d0 tx=0x7ff75802a180 comp rx=0 tx=0).stop 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 -- 172.21.15.129:0/2405850880 shutdown_connections 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 --2- 172.21.15.129:0/2405850880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7640c55a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 -- 172.21.15.129:0/2405850880 >> 172.21.15.129:0/2405850880 conn(0x7ff7640acbf0 msgr2=0x7ff7640ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 -- 172.21.15.129:0/2405850880 shutdown_connections 2024-04-25T01:27:31.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.525+0000 7ff76aa72640 1 -- 172.21.15.129:0/2405850880 wait complete. 2024-04-25T01:27:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.526+0000 7ff76aa72640 1 Processor -- start 2024-04-25T01:27:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.526+0000 7ff76aa72640 1 -- start start 2024-04-25T01:27:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76aa72640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7640c8b20 0x7ff7640ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76aa72640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7640c9030 0x7ff764151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76aa72640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7641eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76aa72640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640c94f0 con 0x7ff7640c8b20 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76aa72640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff764156250 con 0x7ff7640c9030 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76aa72640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7ff76406d3d0 con 0x7ff764152bb0 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76880e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7640c8b20 0x7ff7640ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76880e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7640c8b20 0x7ff7640ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55358/0 (socket says 172.21.15.129:55358) 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76880e640 1 -- 172.21.15.129:0/3773407335 learned_addr learned my addr 172.21.15.129:0/3773407335 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76900f640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7641eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff763fff640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7640c9030 0x7ff764151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.527+0000 7ff76880e640 1 -- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 msgr2=0x7ff7641eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76880e640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff764152bb0 0x7ff7641eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76880e640 1 -- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7640c9030 msgr2=0x7ff764151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76880e640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7640c9030 0x7ff764151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76880e640 1 -- 172.21.15.129:0/3773407335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7640afec0 con 0x7ff7640c8b20 2024-04-25T01:27:31.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76880e640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7640c8b20 0x7ff7640ad8a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff758035040 tx=0x7ff7580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758041070 con 0x7ff7640c8b20 2024-04-25T01:27:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff758004220 con 0x7ff7640c8b20 2024-04-25T01:27:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7640adf00 con 0x7ff7640c8b20 2024-04-25T01:27:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758037040 con 0x7ff7640c8b20 2024-04-25T01:27:31.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.528+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7640b05c0 con 0x7ff7640c8b20 2024-04-25T01:27:31.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.529+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff764068fd0 con 0x7ff7640c8b20 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.529+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff758003610 con 0x7ff7640c8b20 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff7617fa640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff74806be30 0x7ff74806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff7580ab8b0 con 0x7ff7640c8b20 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff763fff640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff74806be30 0x7ff74806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff7617fa640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7ff7480757d0 0x7ff748077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 --> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff758045e50 con 0x7ff7480757d0 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7580b02c0 con 0x7ff7640c8b20 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff76900f640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7ff7480757d0 0x7ff748077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.530+0000 7ff763fff640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff74806be30 0x7ff74806e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff764152550 tx=0x7ff750009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.531+0000 7ff76900f640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7ff7480757d0 0x7ff748077c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.531+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== osd.0 v2:172.21.15.129:6800/1195619493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff758045e50 con 0x7ff7480757d0 2024-04-25T01:27:31.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.544+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 --> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff76406d1b0 con 0x7ff7480757d0 2024-04-25T01:27:31.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff7617fa640 1 -- 172.21.15.129:0/3773407335 <== osd.0 v2:172.21.15.129:6800/1195619493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff76406d1b0 con 0x7ff7480757d0 2024-04-25T01:27:31.525 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:31.525 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:31.525 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7ff7480757d0 msgr2=0x7ff748077c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:6800/1195619493,v1:172.21.15.129:6801/1195619493] conn(0x7ff7480757d0 0x7ff748077c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff74806be30 msgr2=0x7ff74806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7ff74806be30 0x7ff74806e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff764152550 tx=0x7ff750009040 comp rx=0 tx=0).stop 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7640c8b20 msgr2=0x7ff7640ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 --2- 172.21.15.129:0/3773407335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7640c8b20 0x7ff7640ad8a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff758035040 tx=0x7ff7580046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76900f640 1 -- 172.21.15.129:0/3773407335 reap_dead start 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 shutdown_connections 2024-04-25T01:27:31.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 >> 172.21.15.129:0/3773407335 conn(0x7ff7640acbf0 msgr2=0x7ff7640ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:31.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 shutdown_connections 2024-04-25T01:27:31.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.545+0000 7ff76aa72640 1 -- 172.21.15.129:0/3773407335 wait complete. 2024-04-25T01:27:31.535 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T01:27:31.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.653+0000 7f14fc390640 1 Processor -- start 2024-04-25T01:27:31.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.653+0000 7f14fc390640 1 -- start start 2024-04-25T01:27:31.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fc390640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f14f4152fa0 0x7f14f40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fc390640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f41561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fc390640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f40c9860 con 0x7f14f40c7e50 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fc390640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f14f4156720 con 0x7f14f4152fa0 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fc390640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f14f40c5a70 con 0x7f14f40c5520 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14f992b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f41561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fa12c640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f14f4152fa0 0x7f14f40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fa92d640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f14f40c7e50 0x7f14f40c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:44266/0 (socket says 172.21.15.129:44266) 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.654+0000 7f14fa92d640 1 -- 172.21.15.129:0/3048958199 learned_addr learned my addr 172.21.15.129:0/3048958199 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.655+0000 7f14f992b640 1 -- 172.21.15.129:0/3048958199 >> v2:172.21.15.171:3300/0 conn(0x7f14f4152fa0 msgr2=0x7f14f40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.655+0000 7f14f992b640 1 --2- 172.21.15.129:0/3048958199 >> v2:172.21.15.171:3300/0 conn(0x7f14f4152fa0 0x7f14f40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.655+0000 7f14f992b640 1 -- 172.21.15.129:0/3048958199 >> v1:172.21.15.171:6789/0 conn(0x7f14f40c7e50 legacy=0x7f14f40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.655+0000 7f14f992b640 1 -- 172.21.15.129:0/3048958199 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f40c5d50 con 0x7f14f40c5520 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.655+0000 7f14fa12c640 1 --2- 172.21.15.129:0/3048958199 >> v2:172.21.15.171:3300/0 conn(0x7f14f4152fa0 0x7f14f40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.655+0000 7f14f992b640 1 --2- 172.21.15.129:0/3048958199 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f41561e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14f00097a0 tx=0x7f14f002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8fed11edeaf2181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14f912a640 1 -- 172.21.15.129:0/3048958199 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0037070 con 0x7f14f40c5520 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14f912a640 1 -- 172.21.15.129:0/3048958199 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f002adb0 con 0x7f14f40c5520 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14f912a640 1 -- 172.21.15.129:0/3048958199 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0032660 con 0x7f14f40c5520 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 -- 172.21.15.129:0/3048958199 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 msgr2=0x7f14f41561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 --2- 172.21.15.129:0/3048958199 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f41561e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14f00097a0 tx=0x7f14f002a060 comp rx=0 tx=0).stop 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 -- 172.21.15.129:0/3048958199 shutdown_connections 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 --2- 172.21.15.129:0/3048958199 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f41561e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 --2- 172.21.15.129:0/3048958199 >> v2:172.21.15.171:3300/0 conn(0x7f14f4152fa0 0x7f14f40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 -- 172.21.15.129:0/3048958199 >> 172.21.15.129:0/3048958199 conn(0x7f14f406c400 msgr2=0x7f14f406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 -- 172.21.15.129:0/3048958199 shutdown_connections 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.656+0000 7f14fc390640 1 -- 172.21.15.129:0/3048958199 wait complete. 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.657+0000 7f14fc390640 1 Processor -- start 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.657+0000 7f14fc390640 1 -- start start 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fc390640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f4157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fc390640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 0x7f14f41ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fc390640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14f4152fa0 0x7f14f41eea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fc390640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14f40c93d0 con 0x7f14f40c7e50 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fc390640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14f40c6030 con 0x7f14f4152fa0 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fc390640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f14f40c9550 con 0x7f14f40c5520 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 0x7f14f41ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 0x7f14f41ee550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55366/0 (socket says 172.21.15.129:55366) 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 -- 172.21.15.129:0/3824563812 learned_addr learned my addr 172.21.15.129:0/3824563812 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fa92d640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14f4152fa0 0x7f14f41eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14fa12c640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f4157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 -- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 msgr2=0x7f14f4157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f4157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 -- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14f4152fa0 msgr2=0x7f14f41eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.658+0000 7f14f992b640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14f4152fa0 0x7f14f41eea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14f992b640 1 -- 172.21.15.129:0/3824563812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f40c1520 con 0x7f14f40c7e50 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14f992b640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 0x7f14f41ee550 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098c0 tx=0x7f14f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0041030 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f002abb0 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0037040 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f40c1800 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.659+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f40c07b0 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.660+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f4075550 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.660+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14f0044070 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.661+0000 7f14e6ffd640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f14d806be30 0x7f14d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.661+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f14f00ab670 con 0x7f14f40c7e50 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.661+0000 7f14fa12c640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f14d806be30 0x7f14d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.661+0000 7f14fa12c640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f14d806be30 0x7f14d806e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f14f4152590 tx=0x7f14e8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.669+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f14f00795e0 con 0x7f14f40c7e50 2024-04-25T01:27:31.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.852+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f14f40ca0c0 con 0x7f14d806be30 2024-04-25T01:27:31.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.853+0000 7f14e6ffd640 1 -- 172.21.15.129:0/3824563812 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f14f40ca0c0 con 0x7f14d806be30 2024-04-25T01:27:31.833 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T01:27:31.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f14d806be30 msgr2=0x7f14d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f14d806be30 0x7f14d806e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f14f4152590 tx=0x7f14e8009190 comp rx=0 tx=0).stop 2024-04-25T01:27:31.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 msgr2=0x7f14f41ee550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 0x7f14f41ee550 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098c0 tx=0x7f14f00046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 shutdown_connections 2024-04-25T01:27:31.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f14d806be30 0x7f14d806e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14f4152fa0 0x7f14f41eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14f40c7e50 0x7f14f41ee550 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 --2- 172.21.15.129:0/3824563812 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14f40c5520 0x7f14f4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 >> 172.21.15.129:0/3824563812 conn(0x7f14f406c400 msgr2=0x7f14f40ab350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:31.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 shutdown_connections 2024-04-25T01:27:31.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.855+0000 7f14fc390640 1 -- 172.21.15.129:0/3824563812 wait complete. 2024-04-25T01:27:31.846 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T01:27:31.846 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:31.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.968+0000 7fd6d853b640 1 Processor -- start 2024-04-25T01:27:31.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.968+0000 7fd6d853b640 1 -- start start 2024-04-25T01:27:31.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d853b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd6d0152bb0 0x7fd6d00c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d853b640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd6d00c8b20 0x7fd6d015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d853b640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d006cfe0 con 0x7fd6d00c9030 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d853b640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d0155dc0 con 0x7fd6d0152bb0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d853b640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d0155490 con 0x7fd6d00c8b20 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d62d7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd6d0152bb0 0x7fd6d00c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d62d7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fd6d0152bb0 0x7fd6d00c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55378/0 (socket says 172.21.15.129:55378) 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.969+0000 7fd6d62d7640 1 -- 172.21.15.129:0/1710017007 learned_addr learned my addr 172.21.15.129:0/1710017007 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d6ad8640 1 --2- 172.21.15.129:0/1710017007 >> v2:172.21.15.171:3300/0 conn(0x7fd6d00c8b20 0x7fd6d015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d62d7640 1 -- 172.21.15.129:0/1710017007 >> v2:172.21.15.171:3300/0 conn(0x7fd6d00c8b20 msgr2=0x7fd6d015fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d62d7640 1 --2- 172.21.15.129:0/1710017007 >> v2:172.21.15.171:3300/0 conn(0x7fd6d00c8b20 0x7fd6d015fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d62d7640 1 -- 172.21.15.129:0/1710017007 >> v1:172.21.15.171:6789/0 conn(0x7fd6d00c9030 legacy=0x7fd6d0158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d62d7640 1 -- 172.21.15.129:0/1710017007 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d0155770 con 0x7fd6d0152bb0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d6ad8640 1 --2- 172.21.15.129:0/1710017007 >> v2:172.21.15.171:3300/0 conn(0x7fd6d00c8b20 0x7fd6d015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d62d7640 1 --2- 172.21.15.129:0/1710017007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0152bb0 0x7fd6d00c55a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0152f90 tx=0x7fd6c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dfc97214c7e8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.970+0000 7fd6d52d5640 1 -- 172.21.15.129:0/1710017007 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0037070 con 0x7fd6d0152bb0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d52d5640 1 -- 172.21.15.129:0/1710017007 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c0032440 con 0x7fd6d0152bb0 2024-04-25T01:27:31.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d52d5640 1 -- 172.21.15.129:0/1710017007 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0032810 con 0x7fd6d0152bb0 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 -- 172.21.15.129:0/1710017007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0152bb0 msgr2=0x7fd6d00c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 --2- 172.21.15.129:0/1710017007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0152bb0 0x7fd6d00c55a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0152f90 tx=0x7fd6c002a1e0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 -- 172.21.15.129:0/1710017007 shutdown_connections 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 --2- 172.21.15.129:0/1710017007 >> v2:172.21.15.171:3300/0 conn(0x7fd6d00c8b20 0x7fd6d015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 --2- 172.21.15.129:0/1710017007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d0152bb0 0x7fd6d00c55a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 -- 172.21.15.129:0/1710017007 >> 172.21.15.129:0/1710017007 conn(0x7fd6d00acbf0 msgr2=0x7fd6d00ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 -- 172.21.15.129:0/1710017007 shutdown_connections 2024-04-25T01:27:31.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.971+0000 7fd6d853b640 1 -- 172.21.15.129:0/1710017007 wait complete. 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.972+0000 7fd6d853b640 1 Processor -- start 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.972+0000 7fd6d853b640 1 -- start start 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d853b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d00c8b20 0x7fd6d0151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d853b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d00c9030 0x7fd6d006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d853b640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6d0152bb0 0x7fd6d014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d853b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d00c94f0 con 0x7fd6d00c8b20 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d853b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0156250 con 0x7fd6d00c9030 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d853b640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d006d3d0 con 0x7fd6d0152bb0 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d00c8b20 0x7fd6d0151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d00c8b20 0x7fd6d0151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55380/0 (socket says 172.21.15.129:55380) 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 -- 172.21.15.129:0/4197786968 learned_addr learned my addr 172.21.15.129:0/4197786968 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d5ad6640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d00c9030 0x7fd6d006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d6ad8640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6d0152bb0 0x7fd6d014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 -- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6d0152bb0 msgr2=0x7fd6d014a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6d0152bb0 0x7fd6d014a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 -- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d00c9030 msgr2=0x7fd6d006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6d00c9030 0x7fd6d006ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.973+0000 7fd6d62d7640 1 -- 172.21.15.129:0/4197786968 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01ee890 con 0x7fd6d00c8b20 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.974+0000 7fd6d62d7640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d00c8b20 0x7fd6d0151f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000c00 tx=0x7fd6c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.974+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0041020 con 0x7fd6d00c8b20 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.974+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c00306c0 con 0x7fd6d00c8b20 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.974+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01eeb00 con 0x7fd6d00c8b20 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.974+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0037040 con 0x7fd6d00c8b20 2024-04-25T01:27:31.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.974+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01ef0c0 con 0x7fd6d00c8b20 2024-04-25T01:27:31.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.975+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6d0068fd0 con 0x7fd6d00c8b20 2024-04-25T01:27:31.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.975+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6c0044020 con 0x7fd6d00c8b20 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6c6ffd640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd6b406be60 0x7fd6b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd6c00ab900 con 0x7fd6d00c8b20 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6d5ad6640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd6b406be60 0x7fd6b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6c6ffd640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7fd6b4075870 0x7fd6b4077d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 --> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6c0032440 con 0x7fd6b4075870 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6c00b02c0 con 0x7fd6d00c8b20 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6d6ad8640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7fd6b4075870 0x7fd6b4077d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6d5ad6640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd6b406be60 0x7fd6b406e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00ab330 tx=0x7fd6cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.976+0000 7fd6d6ad8640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7fd6b4075870 0x7fd6b4077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:31.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.977+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== osd.1 v2:172.21.15.129:6816/3197437953 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6c0032440 con 0x7fd6b4075870 2024-04-25T01:27:31.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.990+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 --> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd6d014c300 con 0x7fd6b4075870 2024-04-25T01:27:31.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.990+0000 7fd6c6ffd640 1 -- 172.21.15.129:0/4197786968 <== osd.1 v2:172.21.15.129:6816/3197437953 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd6d014c300 con 0x7fd6b4075870 2024-04-25T01:27:31.970 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:31.971 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:31.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.990+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7fd6b4075870 msgr2=0x7fd6b4077d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.990+0000 7fd6d853b640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:6816/3197437953,v1:172.21.15.129:6817/3197437953] conn(0x7fd6b4075870 0x7fd6b4077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd6b406be60 msgr2=0x7fd6b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd6b406be60 0x7fd6b406e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00ab330 tx=0x7fd6cc009040 comp rx=0 tx=0).stop 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d00c8b20 msgr2=0x7fd6d0151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 --2- 172.21.15.129:0/4197786968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6d00c8b20 0x7fd6d0151f60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000c00 tx=0x7fd6c00046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d6ad8640 1 -- 172.21.15.129:0/4197786968 reap_dead start 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 shutdown_connections 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 >> 172.21.15.129:0/4197786968 conn(0x7fd6d00acbf0 msgr2=0x7fd6d00aa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 shutdown_connections 2024-04-25T01:27:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:31.991+0000 7fd6d853b640 1 -- 172.21.15.129:0/4197786968 wait complete. 2024-04-25T01:27:31.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T01:27:32.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.099+0000 7f54d264c640 1 Processor -- start 2024-04-25T01:27:32.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.099+0000 7f54d264c640 1 -- start start 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d264c640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f54cc152fa0 0x7f54cc0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d264c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc1561e0 0x7f54cc0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d264c640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54cc0c9860 con 0x7f54cc0c5520 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d264c640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f54cc0c5a70 con 0x7f54cc152fa0 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d264c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f54cc0c5d50 con 0x7f54cc1561e0 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54cb7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f54cc0c5520 0x7f54cc0c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60126/0 (socket says 172.21.15.129:60126) 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54cb7fe640 1 -- 172.21.15.129:0/431478488 learned_addr learned my addr 172.21.15.129:0/431478488 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54cbfff640 1 --2- 172.21.15.129:0/431478488 >> v2:172.21.15.171:3300/0 conn(0x7f54cc152fa0 0x7f54cc0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d0be9640 1 --2- 172.21.15.129:0/431478488 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc1561e0 0x7f54cc0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54caffd640 1 -- 172.21.15.129:0/431478488 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210805485 0 0) 0x7f54bc001110 con 0x7f54cc0c5520 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54caffd640 1 -- 172.21.15.129:0/431478488 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54b4003520 con 0x7f54cc0c5520 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d0be9640 1 -- 172.21.15.129:0/431478488 >> v2:172.21.15.171:3300/0 conn(0x7f54cc152fa0 msgr2=0x7f54cc0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d0be9640 1 --2- 172.21.15.129:0/431478488 >> v2:172.21.15.171:3300/0 conn(0x7f54cc152fa0 0x7f54cc0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d0be9640 1 -- 172.21.15.129:0/431478488 >> v1:172.21.15.129:6789/0 conn(0x7f54cc0c5520 legacy=0x7f54cc0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54caffd640 1 -- 172.21.15.129:0/431478488 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3074665233 0 0) 0x7f54b4003520 con 0x7f54cc0c5520 2024-04-25T01:27:32.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54d0be9640 1 -- 172.21.15.129:0/431478488 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc0c7650 con 0x7f54cc1561e0 2024-04-25T01:27:32.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.100+0000 7f54cbfff640 1 --2- 172.21.15.129:0/431478488 >> v2:172.21.15.171:3300/0 conn(0x7f54cc152fa0 0x7f54cc0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:32.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.101+0000 7f54d0be9640 1 --2- 172.21.15.129:0/431478488 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc1561e0 0x7f54cc0c8cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009d90 tx=0x7f54c402a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b2359228ed6219d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.101+0000 7f54caffd640 1 -- 172.21.15.129:0/431478488 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc001840 con 0x7f54cc1561e0 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.101+0000 7f54caffd640 1 -- 172.21.15.129:0/431478488 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4007540 con 0x7f54cc1561e0 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.101+0000 7f54caffd640 1 -- 172.21.15.129:0/431478488 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4007910 con 0x7f54cc1561e0 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 -- 172.21.15.129:0/431478488 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc1561e0 msgr2=0x7f54cc0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 --2- 172.21.15.129:0/431478488 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc1561e0 0x7f54cc0c8cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009d90 tx=0x7f54c402a4d0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 -- 172.21.15.129:0/431478488 shutdown_connections 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 --2- 172.21.15.129:0/431478488 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc1561e0 0x7f54cc0c8cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 --2- 172.21.15.129:0/431478488 >> v2:172.21.15.171:3300/0 conn(0x7f54cc152fa0 0x7f54cc0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 -- 172.21.15.129:0/431478488 >> 172.21.15.129:0/431478488 conn(0x7f54cc06c400 msgr2=0x7f54cc06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 -- 172.21.15.129:0/431478488 shutdown_connections 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.102+0000 7f54d264c640 1 -- 172.21.15.129:0/431478488 wait complete. 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 Processor -- start 2024-04-25T01:27:32.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 -- start start 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc0c5520 0x7f54cc1eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 0x7f54cc1ef060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54cc1561e0 0x7f54cc150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc0c9200 con 0x7f54cc152fa0 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.103+0000 7f54d264c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc0c6030 con 0x7f54cc1561e0 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54d264c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f54cc0c9380 con 0x7f54cc0c5520 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 0x7f54cc1ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 0x7f54cc1ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55382/0 (socket says 172.21.15.129:55382) 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 -- 172.21.15.129:0/1842404365 learned_addr learned my addr 172.21.15.129:0/1842404365 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cbfff640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc0c5520 0x7f54cc1eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54d0be9640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54cc1561e0 0x7f54cc150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 -- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc0c5520 msgr2=0x7f54cc1eeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc0c5520 0x7f54cc1eeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 -- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54cc1561e0 msgr2=0x7f54cc150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54cc1561e0 0x7f54cc150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 -- 172.21.15.129:0/1842404365 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc158770 con 0x7f54cc152fa0 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54cb7fe640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 0x7f54cc1ef060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00c730 tx=0x7f54bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc017070 con 0x7f54cc152fa0 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54bc005af0 con 0x7f54cc152fa0 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc007070 con 0x7f54cc152fa0 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54cc158a50 con 0x7f54cc152fa0 2024-04-25T01:27:32.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.104+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54cc0c18b0 con 0x7f54cc152fa0 2024-04-25T01:27:32.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.105+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54cc0c8770 con 0x7f54cc152fa0 2024-04-25T01:27:32.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.106+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54bc01a070 con 0x7f54cc152fa0 2024-04-25T01:27:32.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.106+0000 7f54c8ff9640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f54b806be30 0x7f54b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.106+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f54bc08a5c0 con 0x7f54cc152fa0 2024-04-25T01:27:32.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.106+0000 7f54cbfff640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f54b806be30 0x7f54b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.107+0000 7f54cbfff640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f54b806be30 0x7f54b806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0c3ab0 tx=0x7f54c0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.111+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54cc0c8770 con 0x7f54cc152fa0 2024-04-25T01:27:32.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.290+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f54cc1efa10 con 0x7f54b806be30 2024-04-25T01:27:32.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.290+0000 7f54c8ff9640 1 -- 172.21.15.129:0/1842404365 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f54cc1efa10 con 0x7f54b806be30 2024-04-25T01:27:32.270 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f54b806be30 msgr2=0x7f54b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f54b806be30 0x7f54b806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0c3ab0 tx=0x7f54c0009190 comp rx=0 tx=0).stop 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 msgr2=0x7f54cc1ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 0x7f54cc1ef060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00c730 tx=0x7f54bc00cbf0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 shutdown_connections 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f54b806be30 0x7f54b806e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54cc1561e0 0x7f54cc150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54cc152fa0 0x7f54cc1ef060 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 --2- 172.21.15.129:0/1842404365 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f54cc0c5520 0x7f54cc1eeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 >> 172.21.15.129:0/1842404365 conn(0x7f54cc06c400 msgr2=0x7f54cc06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 shutdown_connections 2024-04-25T01:27:32.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.293+0000 7f54d264c640 1 -- 172.21.15.129:0/1842404365 wait complete. 2024-04-25T01:27:32.283 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T01:27:32.284 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.417+0000 7f9530b90640 1 Processor -- start 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.417+0000 7f9530b90640 1 -- start start 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f9530b90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 0x7f952c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f9530b90640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952c0c6100 con 0x7f952c0c1ca0 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f9530b90640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952c0c63e0 con 0x7f952c0c4090 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f9530b90640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f952c0c5a00 con 0x7f952c0c3590 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f952ad9d640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f952c0c1ca0 0x7f952c16cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60140/0 (socket says 172.21.15.129:60140) 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f952ad9d640 1 -- 172.21.15.129:0/598539876 learned_addr learned my addr 172.21.15.129:0/598539876 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f952a59c640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 0x7f952c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.418+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197675681 0 0) 0x7f952c0c6100 con 0x7f952c0c1ca0 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9518003500 con 0x7f952c0c1ca0 2024-04-25T01:27:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876032012 0 0) 0x7f952c0c63e0 con 0x7f952c0c4090 2024-04-25T01:27:32.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f952c0c6100 con 0x7f952c0c4090 2024-04-25T01:27:32.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 301765374 0 0) 0x7f9518003500 con 0x7f952c0c1ca0 2024-04-25T01:27:32.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 msgr2=0x7f952c0ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:32.412 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 0x7f952c0ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 >> v1:172.21.15.171:6789/0 conn(0x7f952c0c4090 legacy=0x7f952c0c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c165850 con 0x7f952c0c1ca0 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.428+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f95240019f0 con 0x7f952c0c1ca0 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 >> v1:172.21.15.129:6789/0 conn(0x7f952c0c1ca0 legacy=0x7f952c16cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 0x7f9518003890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518003dd0 0x7f9518004250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9518004da0 0x7f9518005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9518005aa0 con 0x7f9518004da0 2024-04-25T01:27:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9518005c50 con 0x7f9518003dd0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f9518005e50 con 0x7f952c0c3590 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952ad9d640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9518004da0 0x7f9518005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952a59c640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 0x7f9518003890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f9523fff640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518003dd0 0x7f9518004250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952ad9d640 1 -- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 msgr2=0x7f9518003890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952ad9d640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f952c0c3590 0x7f9518003890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952ad9d640 1 -- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518003dd0 msgr2=0x7f9518004250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952ad9d640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9518003dd0 0x7f9518004250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.429+0000 7f952ad9d640 1 -- 172.21.15.129:0/598539876 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9518003500 con 0x7f9518004da0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f952ad9d640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9518004da0 0x7f9518005220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f952400a820 tx=0x7f951800afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab60cece93d2302c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952400a450 con 0x7f9518004da0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952402f070 con 0x7f9518004da0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9529d9b640 1 -- 172.21.15.129:0/598539876 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952402bc50 con 0x7f9518004da0 2024-04-25T01:27:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9530b90640 1 -- 172.21.15.129:0/598539876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9518004da0 msgr2=0x7f9518005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9530b90640 1 --2- 172.21.15.129:0/598539876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9518004da0 0x7f9518005220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f952400a820 tx=0x7f951800afc0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f952ad9d640 1 -- 172.21.15.129:0/598539876 reap_dead start 2024-04-25T01:27:32.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9530b90640 1 -- 172.21.15.129:0/598539876 shutdown_connections 2024-04-25T01:27:32.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9530b90640 1 -- 172.21.15.129:0/598539876 >> 172.21.15.129:0/598539876 conn(0x7f952c0c6890 msgr2=0x7f952c0c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9530b90640 1 -- 172.21.15.129:0/598539876 shutdown_connections 2024-04-25T01:27:32.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.430+0000 7f9530b90640 1 -- 172.21.15.129:0/598539876 wait complete. 2024-04-25T01:27:32.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 Processor -- start 2024-04-25T01:27:32.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 -- start start 2024-04-25T01:27:32.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9518004da0 0x7f952c1f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f952c070690 0x7f952c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f952c072b40 0x7f952c1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f952c0a5f70 con 0x7f952c070690 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f952c08b350 con 0x7f952c072b40 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9530b90640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f952c0a60f0 con 0x7f9518004da0 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9523fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f952c070690 0x7f952c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9523fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f952c070690 0x7f952c071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55398/0 (socket says 172.21.15.129:55398) 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f9523fff640 1 -- 172.21.15.129:0/2733920054 learned_addr learned my addr 172.21.15.129:0/2733920054 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f952a59c640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9518004da0 0x7f952c1f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.432+0000 7f952ad9d640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f952c072b40 0x7f952c1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9523fff640 1 -- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9518004da0 msgr2=0x7f952c1f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9523fff640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9518004da0 0x7f952c1f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9523fff640 1 -- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f952c072b40 msgr2=0x7f952c1f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9523fff640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f952c072b40 0x7f952c1f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9523fff640 1 -- 172.21.15.129:0/2733920054 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c070c00 con 0x7f952c070690 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9523fff640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f952c070690 0x7f952c071ca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95140010e0 tx=0x7f951400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951400fea0 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9514018070 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952c070ee0 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951400d6b0 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.433+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952c0714a0 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.434+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f952c068fd0 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f951400d850 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f9522ffd640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f950c06be60 0x7f950c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f951408af60 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f9522ffd640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f950c075870 0x7f950c077d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 --> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9514009e50 con 0x7f950c075870 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f951408d600 con 0x7f952c070690 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.435+0000 7f952a59c640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f950c06be60 0x7f950c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.436+0000 7f952ad9d640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f950c075870 0x7f950c077d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.436+0000 7f952ad9d640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f950c075870 0x7f950c077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.436+0000 7f952a59c640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f950c06be60 0x7f950c06e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f951c0094d0 tx=0x7f951c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.437+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== osd.2 v2:172.21.15.129:6824/1981636327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9514009e50 con 0x7f950c075870 2024-04-25T01:27:32.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.450+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 --> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f952c163fb0 con 0x7f950c075870 2024-04-25T01:27:32.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.450+0000 7f9522ffd640 1 -- 172.21.15.129:0/2733920054 <== osd.2 v2:172.21.15.129:6824/1981636327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f952c163fb0 con 0x7f950c075870 2024-04-25T01:27:32.430 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:32.431 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:32.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.450+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f950c075870 msgr2=0x7f950c077d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:6824/1981636327,v1:172.21.15.129:6825/1981636327] conn(0x7f950c075870 0x7f950c077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f950c06be60 msgr2=0x7f950c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f950c06be60 0x7f950c06e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f951c0094d0 tx=0x7f951c00b040 comp rx=0 tx=0).stop 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f952c070690 msgr2=0x7f952c071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 --2- 172.21.15.129:0/2733920054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f952c070690 0x7f952c071ca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95140010e0 tx=0x7f951400fc50 comp rx=0 tx=0).stop 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f952ad9d640 1 -- 172.21.15.129:0/2733920054 reap_dead start 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 shutdown_connections 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 >> 172.21.15.129:0/2733920054 conn(0x7f952c0c6890 msgr2=0x7f952c0a8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 shutdown_connections 2024-04-25T01:27:32.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.451+0000 7f9530b90640 1 -- 172.21.15.129:0/2733920054 wait complete. 2024-04-25T01:27:32.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T01:27:32.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.561+0000 7f723a76a640 1 Processor -- start 2024-04-25T01:27:32.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.561+0000 7f723a76a640 1 -- start start 2024-04-25T01:27:32.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f723a76a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f72340c5520 0x7f72340ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f723a76a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340c9860 con 0x7f72341561e0 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f723a76a640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340c5a70 con 0x7f7234152fa0 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f723a76a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f72340c5d50 con 0x7f72340c5520 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f7238d07640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f72341561e0 0x7f72340c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60152/0 (socket says 172.21.15.129:60152) 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f7238d07640 1 -- 172.21.15.129:0/2878575072 learned_addr learned my addr 172.21.15.129:0/2878575072 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f72337fe640 1 --2- 172.21.15.129:0/2878575072 >> v2:172.21.15.171:3300/0 conn(0x7f72340c5520 0x7f72340ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f7232ffd640 1 -- 172.21.15.129:0/2878575072 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093973651 0 0) 0x7f722c001490 con 0x7f72341561e0 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.562+0000 7f7232ffd640 1 -- 172.21.15.129:0/2878575072 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f721c003520 con 0x7f72341561e0 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.563+0000 7f72337fe640 1 -- 172.21.15.129:0/2878575072 >> v1:172.21.15.171:6789/0 conn(0x7f7234152fa0 legacy=0x7f72340c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.563+0000 7f7232ffd640 1 -- 172.21.15.129:0/2878575072 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3117353566 0 0) 0x7f72340c5a70 con 0x7f7234152fa0 2024-04-25T01:27:32.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.563+0000 7f72337fe640 1 -- 172.21.15.129:0/2878575072 >> v1:172.21.15.129:6789/0 conn(0x7f72341561e0 legacy=0x7f72340c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.563+0000 7f72337fe640 1 -- 172.21.15.129:0/2878575072 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340c7650 con 0x7f72340c5520 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.563+0000 7f72337fe640 1 --2- 172.21.15.129:0/2878575072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72340ca2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f72280097a0 tx=0x7f722802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9e13901fb66051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.563+0000 7f7232ffd640 1 -- 172.21.15.129:0/2878575072 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228037070 con 0x7f72340c5520 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f7232ffd640 1 -- 172.21.15.129:0/2878575072 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7228032440 con 0x7f72340c5520 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f7232ffd640 1 -- 172.21.15.129:0/2878575072 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722802ae60 con 0x7f72340c5520 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 -- 172.21.15.129:0/2878575072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 msgr2=0x7f72340ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 --2- 172.21.15.129:0/2878575072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72340ca2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f72280097a0 tx=0x7f722802a110 comp rx=0 tx=0).stop 2024-04-25T01:27:32.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 -- 172.21.15.129:0/2878575072 shutdown_connections 2024-04-25T01:27:32.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 --2- 172.21.15.129:0/2878575072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72340ca2e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 -- 172.21.15.129:0/2878575072 >> 172.21.15.129:0/2878575072 conn(0x7f723406c400 msgr2=0x7f723406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 -- 172.21.15.129:0/2878575072 shutdown_connections 2024-04-25T01:27:32.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.564+0000 7f723a76a640 1 -- 172.21.15.129:0/2878575072 wait complete. 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.565+0000 7f723a76a640 1 Processor -- start 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.565+0000 7f723a76a640 1 -- start start 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f723a76a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72341ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f723a76a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7234152fa0 0x7f72341e2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f723a76a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 0x7f72340bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f723a76a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72340c93d0 con 0x7f72341561e0 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f723a76a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72340c6030 con 0x7f72340c5520 2024-04-25T01:27:32.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f723a76a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f72340c9550 con 0x7f7234152fa0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 0x7f72340bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 0x7f72340bfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55402/0 (socket says 172.21.15.129:55402) 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 -- 172.21.15.129:0/2641276739 learned_addr learned my addr 172.21.15.129:0/2641276739 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7233fff640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72341ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f72337fe640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7234152fa0 0x7f72341e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 -- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7234152fa0 msgr2=0x7f72341e2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7234152fa0 0x7f72341e2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 -- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 msgr2=0x7f72341ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72341ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.566+0000 7f7238d07640 1 -- 172.21.15.129:0/2641276739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72341ec360 con 0x7f72341561e0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.567+0000 7f7238d07640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 0x7f72340bfe70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f722c00cb80 tx=0x7f722c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.567+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c017070 con 0x7f72341561e0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.567+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c00ad40 con 0x7f72341561e0 2024-04-25T01:27:32.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.567+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72340c42e0 con 0x7f72341561e0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.567+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c012620 con 0x7f72341561e0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.567+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72340c48a0 con 0x7f72341561e0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.568+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72340ad4f0 con 0x7f72341561e0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.569+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f722c01a070 con 0x7f72341561e0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.569+0000 7f7230ff9640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f722006be60 0x7f722006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.569+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f722c08af50 con 0x7f72341561e0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.569+0000 7f7233fff640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f722006be60 0x7f722006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.570+0000 7f7233fff640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f722006be60 0x7f722006e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7224000f60 tx=0x7f7224009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.574+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72340ad4f0 con 0x7f72341561e0 2024-04-25T01:27:32.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.756+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f723406d480 con 0x7f722006be60 2024-04-25T01:27:32.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.756+0000 7f7230ff9640 1 -- 172.21.15.129:0/2641276739 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f723406d480 con 0x7f722006be60 2024-04-25T01:27:32.736 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f722006be60 msgr2=0x7f722006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f722006be60 0x7f722006e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7224000f60 tx=0x7f7224009040 comp rx=0 tx=0).stop 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 msgr2=0x7f72340bfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 0x7f72340bfe70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f722c00cb80 tx=0x7f722c00a430 comp rx=0 tx=0).stop 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 shutdown_connections 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f722006be60 0x7f722006e320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7234152fa0 0x7f72341e2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72341561e0 0x7f72340bfe70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 --2- 172.21.15.129:0/2641276739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72340c5520 0x7f72341ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 >> 172.21.15.129:0/2641276739 conn(0x7f723406c400 msgr2=0x7f72340c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 shutdown_connections 2024-04-25T01:27:32.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.758+0000 7f723a76a640 1 -- 172.21.15.129:0/2641276739 wait complete. 2024-04-25T01:27:32.749 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T01:27:32.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:32.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.872+0000 7f3afe2b3640 1 Processor -- start 2024-04-25T01:27:32.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.872+0000 7f3afe2b3640 1 -- start start 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afe2b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af8152bb0 0x7f3af80c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afe2b3640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3af80c8b20 0x7f3af815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afe2b3640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af806cfe0 con 0x7f3af80c9030 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afe2b3640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f3af8155dc0 con 0x7f3af80c8b20 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afe2b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f3af8155490 con 0x7f3af8152bb0 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3af77fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f3af80c9030 0x7f3af8158970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60154/0 (socket says 172.21.15.129:60154) 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3af77fe640 1 -- 172.21.15.129:0/1284114980 learned_addr learned my addr 172.21.15.129:0/1284114980 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3af7fff640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af8152bb0 0x7f3af80c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afc850640 1 --2- 172.21.15.129:0/1284114980 >> v2:172.21.15.171:3300/0 conn(0x7f3af80c8b20 0x7f3af815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3af6ffd640 1 -- 172.21.15.129:0/1284114980 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002175253 0 0) 0x7f3af806cfe0 con 0x7f3af80c9030 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3af6ffd640 1 -- 172.21.15.129:0/1284114980 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ae0003520 con 0x7f3af80c9030 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afc850640 1 -- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af8152bb0 msgr2=0x7f3af80c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afc850640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af8152bb0 0x7f3af80c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afc850640 1 -- 172.21.15.129:0/1284114980 >> v1:172.21.15.129:6789/0 conn(0x7f3af80c9030 legacy=0x7f3af8158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3afc850640 1 -- 172.21.15.129:0/1284114980 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af8155770 con 0x7f3af80c8b20 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.873+0000 7f3af7fff640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af8152bb0 0x7f3af80c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.874+0000 7f3afc850640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af80c8b20 0x7f3af815fd00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009d90 tx=0x7f3af002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6c323dfefeff00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.874+0000 7f3af6ffd640 1 -- 172.21.15.129:0/1284114980 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0031600 con 0x7f3af80c8b20 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.874+0000 7f3af6ffd640 1 -- 172.21.15.129:0/1284114980 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af00317a0 con 0x7f3af80c8b20 2024-04-25T01:27:32.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.874+0000 7f3af6ffd640 1 -- 172.21.15.129:0/1284114980 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0031b70 con 0x7f3af80c8b20 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 -- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af80c8b20 msgr2=0x7f3af815fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af80c8b20 0x7f3af815fd00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009d90 tx=0x7f3af002a5d0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 -- 172.21.15.129:0/1284114980 shutdown_connections 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af80c8b20 0x7f3af815fd00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 --2- 172.21.15.129:0/1284114980 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af8152bb0 0x7f3af80c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 -- 172.21.15.129:0/1284114980 >> 172.21.15.129:0/1284114980 conn(0x7f3af80acbf0 msgr2=0x7f3af80ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 -- 172.21.15.129:0/1284114980 shutdown_connections 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.875+0000 7f3afe2b3640 1 -- 172.21.15.129:0/1284114980 wait complete. 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.876+0000 7f3afe2b3640 1 Processor -- start 2024-04-25T01:27:32.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.876+0000 7f3afe2b3640 1 -- start start 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afe2b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af80c8b20 0x7f3af80be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afe2b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80c9030 0x7f3af81eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afe2b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af8152bb0 0x7f3af81ee530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afe2b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3af80c94f0 con 0x7f3af80c9030 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afe2b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3af8156250 con 0x7f3af8152bb0 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afe2b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f3af806d3d0 con 0x7f3af80c8b20 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80c9030 0x7f3af81eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80c9030 0x7f3af81eb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55416/0 (socket says 172.21.15.129:55416) 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af77fe640 1 -- 172.21.15.129:0/3336638371 learned_addr learned my addr 172.21.15.129:0/3336638371 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af80c8b20 0x7f3af80be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3afc850640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af8152bb0 0x7f3af81ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 -- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af8152bb0 msgr2=0x7f3af81ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3af8152bb0 0x7f3af81ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 -- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80c9030 msgr2=0x7f3af81eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80c9030 0x7f3af81eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 -- 172.21.15.129:0/3336638371 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af81480c0 con 0x7f3af80c8b20 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af77fe640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3af80c9030 0x7f3af81eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af7fff640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af80c8b20 0x7f3af80be9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00e8a0 tx=0x7f3aec00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec017070 con 0x7f3af80c8b20 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.877+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aec00c940 con 0x7f3af80c8b20 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.878+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af8151ed0 con 0x7f3af80c8b20 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.878+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00cd30 con 0x7f3af80c8b20 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.878+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af80bef00 con 0x7f3af80c8b20 2024-04-25T01:27:32.860 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.878+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3af80c01c0 con 0x7f3af80c8b20 2024-04-25T01:27:32.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.879+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3aec01a070 con 0x7f3af80c8b20 2024-04-25T01:27:32.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.879+0000 7f3af4ff9640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3ae406be60 0x7f3ae406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.879+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3aec08b4a0 con 0x7f3af80c8b20 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.879+0000 7f3af4ff9640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f3ae4075870 0x7f3ae4077d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.879+0000 7f3af77fe640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3ae406be60 0x7f3ae406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.880+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 --> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3aec023e70 con 0x7f3ae4075870 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.880+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3aec08b930 con 0x7f3af80c8b20 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.880+0000 7f3afc850640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f3ae4075870 0x7f3ae4077d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.880+0000 7f3afc850640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f3ae4075870 0x7f3ae4077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.880+0000 7f3af77fe640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3ae406be60 0x7f3ae406e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3af806bd80 tx=0x7f3ae8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.881+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== osd.3 v2:172.21.15.129:6808/4291613316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3aec023e70 con 0x7f3ae4075870 2024-04-25T01:27:32.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 --> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3af8147e70 con 0x7f3ae4075870 2024-04-25T01:27:32.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3af4ff9640 1 -- 172.21.15.129:0/3336638371 <== osd.3 v2:172.21.15.129:6808/4291613316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3af8147e70 con 0x7f3ae4075870 2024-04-25T01:27:32.874 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:32.874 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:32.874 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:32.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f3ae4075870 msgr2=0x7f3ae4077d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.129:6808/4291613316,v1:172.21.15.129:6809/4291613316] conn(0x7f3ae4075870 0x7f3ae4077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3ae406be60 msgr2=0x7f3ae406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f3ae406be60 0x7f3ae406e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3af806bd80 tx=0x7f3ae8009040 comp rx=0 tx=0).stop 2024-04-25T01:27:32.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af80c8b20 msgr2=0x7f3af80be9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 --2- 172.21.15.129:0/3336638371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3af80c8b20 0x7f3af80be9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00e8a0 tx=0x7f3aec00ed60 comp rx=0 tx=0).stop 2024-04-25T01:27:32.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afc850640 1 -- 172.21.15.129:0/3336638371 reap_dead start 2024-04-25T01:27:32.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 shutdown_connections 2024-04-25T01:27:32.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.894+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 >> 172.21.15.129:0/3336638371 conn(0x7f3af80acbf0 msgr2=0x7f3af8153020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.895+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 shutdown_connections 2024-04-25T01:27:32.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:32.895+0000 7f3afe2b3640 1 -- 172.21.15.129:0/3336638371 wait complete. 2024-04-25T01:27:32.885 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T01:27:32.989 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.009+0000 7f2e6a9e7640 1 Processor -- start 2024-04-25T01:27:32.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.009+0000 7f2e6a9e7640 1 -- start start 2024-04-25T01:27:32.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e6a9e7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2e640a7d00 0x7f2e640c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e6a9e7640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e6a9e7640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e640a8160 con 0x7f2e640c4940 2024-04-25T01:27:32.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e6a9e7640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6406fa70 con 0x7f2e640a7d00 2024-04-25T01:27:32.991 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e6a9e7640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f2e640c4d20 con 0x7f2e640c38f0 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e637fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2e640a7d00 0x7f2e640c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e637fe640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f2e640a7d00 0x7f2e640c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55418/0 (socket says 172.21.15.129:55418) 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e637fe640 1 -- 172.21.15.129:0/2295922424 learned_addr learned my addr 172.21.15.129:0/2295922424 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e63fff640 1 --1- 172.21.15.129:0/2295922424 >> v1:172.21.15.129:6789/0 conn(0x7f2e640c4940 0x7f2e640aa6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60168/0 (socket says 172.21.15.129:60168) 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.010+0000 7f2e68f84640 1 --2- 172.21.15.129:0/2295922424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.011+0000 7f2e637fe640 1 -- 172.21.15.129:0/2295922424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 msgr2=0x7f2e64160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.011+0000 7f2e637fe640 1 --2- 172.21.15.129:0/2295922424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.011+0000 7f2e637fe640 1 -- 172.21.15.129:0/2295922424 >> v1:172.21.15.129:6789/0 conn(0x7f2e640c4940 legacy=0x7f2e640aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.011+0000 7f2e62ffd640 1 -- 172.21.15.129:0/2295922424 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955842069 0 0) 0x7f2e640a8160 con 0x7f2e640c4940 2024-04-25T01:27:32.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.011+0000 7f2e637fe640 1 -- 172.21.15.129:0/2295922424 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e640c3d80 con 0x7f2e640a7d00 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e637fe640 1 --2- 172.21.15.129:0/2295922424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e640c33b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c8c6edebd80f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e62ffd640 1 -- 172.21.15.129:0/2295922424 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58037070 con 0x7f2e640a7d00 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e62ffd640 1 -- 172.21.15.129:0/2295922424 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e58032440 con 0x7f2e640a7d00 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e62ffd640 1 -- 172.21.15.129:0/2295922424 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58032810 con 0x7f2e640a7d00 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/2295922424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 msgr2=0x7f2e640c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/2295922424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e640c33b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5802a1c0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/2295922424 shutdown_connections 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/2295922424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/2295922424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e640c33b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/2295922424 >> 172.21.15.129:0/2295922424 conn(0x7f2e64066ea0 msgr2=0x7f2e640672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/2295922424 shutdown_connections 2024-04-25T01:27:32.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.012+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/2295922424 wait complete. 2024-04-25T01:27:32.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 Processor -- start 2024-04-25T01:27:32.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 -- start start 2024-04-25T01:27:32.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e641ee400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e640c4940 0x7f2e641514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64070e90 con 0x7f2e640a7d00 2024-04-25T01:27:32.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640c2d30 con 0x7f2e640c4940 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e6a9e7640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f2e64071170 con 0x7f2e640c38f0 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.014+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e641ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e641ee400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55426/0 (socket says 172.21.15.129:55426) 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 -- 172.21.15.129:0/736258709 learned_addr learned my addr 172.21.15.129:0/736258709 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e637fe640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e68f84640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e640c4940 0x7f2e641514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 -- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 msgr2=0x7f2e64150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 -- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e640c4940 msgr2=0x7f2e641514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e640c4940 0x7f2e641514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 -- 172.21.15.129:0/736258709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e641eec40 con 0x7f2e640a7d00 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e63fff640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e641ee400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c4d0 tx=0x7f2e5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:32.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54018070 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e54007500 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641eeee0 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54004a20 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.015+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e64157bb0 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.016+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e641ef0b0 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.017+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e54004bc0 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.017+0000 7f2e60ff9640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f2e5006be30 0x7f2e5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.017+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2e5408a500 con 0x7f2e640a7d00 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.017+0000 7f2e637fe640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f2e5006be30 0x7f2e5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:32.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.018+0000 7f2e637fe640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f2e5006be30 0x7f2e5006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e58034000 tx=0x7f2e5802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.025+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2e540583e0 con 0x7f2e640a7d00 2024-04-25T01:27:33.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.208+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f2e64075550 con 0x7f2e5006be30 2024-04-25T01:27:33.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.209+0000 7f2e60ff9640 1 -- 172.21.15.129:0/736258709 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2e64075550 con 0x7f2e5006be30 2024-04-25T01:27:33.189 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T01:27:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f2e5006be30 msgr2=0x7f2e5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f2e5006be30 0x7f2e5006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e58034000 tx=0x7f2e5802e000 comp rx=0 tx=0).stop 2024-04-25T01:27:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 msgr2=0x7f2e641ee400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e641ee400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c4d0 tx=0x7f2e5400c990 comp rx=0 tx=0).stop 2024-04-25T01:27:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 shutdown_connections 2024-04-25T01:27:33.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f2e5006be30 0x7f2e5006e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e640c38f0 0x7f2e64150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e640c4940 0x7f2e641514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 --2- 172.21.15.129:0/736258709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e640a7d00 0x7f2e641ee400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 >> 172.21.15.129:0/736258709 conn(0x7f2e64066ea0 msgr2=0x7f2e641551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 shutdown_connections 2024-04-25T01:27:33.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.211+0000 7f2e6a9e7640 1 -- 172.21.15.129:0/736258709 wait complete. 2024-04-25T01:27:33.202 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T01:27:33.202 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:33.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.324+0000 7f449141c640 1 Processor -- start 2024-04-25T01:27:33.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.324+0000 7f449141c640 1 -- start start 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f449141c640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f448c0c9030 0x7f448c158970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f449141c640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f448c0c8b20 0x7f448c158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f449141c640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448c06cfe0 con 0x7f448c152bb0 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f449141c640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f448c1593f0 con 0x7f448c0c9030 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f449141c640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f448c155dc0 con 0x7f448c0c8b20 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f448c0c9030 0x7f448c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f448c0c9030 0x7f448c158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55438/0 (socket says 172.21.15.129:55438) 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 -- 172.21.15.129:0/3712160252 learned_addr learned my addr 172.21.15.129:0/3712160252 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448b7fe640 1 --2- 172.21.15.129:0/3712160252 >> v2:172.21.15.171:3300/0 conn(0x7f448c0c8b20 0x7f448c158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 -- 172.21.15.129:0/3712160252 >> v2:172.21.15.171:3300/0 conn(0x7f448c0c8b20 msgr2=0x7f448c158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 --2- 172.21.15.129:0/3712160252 >> v2:172.21.15.171:3300/0 conn(0x7f448c0c8b20 0x7f448c158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 -- 172.21.15.129:0/3712160252 >> v1:172.21.15.171:6789/0 conn(0x7f448c152bb0 legacy=0x7f448c0c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.325+0000 7f448a7fc640 1 -- 172.21.15.129:0/3712160252 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c155490 con 0x7f448c0c9030 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.326+0000 7f448a7fc640 1 --2- 172.21.15.129:0/3712160252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c0c9030 0x7f448c158970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f448c152f90 tx=0x7f447c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66163abee2a8b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.326+0000 7f4489ffb640 1 -- 172.21.15.129:0/3712160252 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c037070 con 0x7f448c0c9030 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.326+0000 7f4489ffb640 1 -- 172.21.15.129:0/3712160252 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447c032440 con 0x7f448c0c9030 2024-04-25T01:27:33.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.326+0000 7f4489ffb640 1 -- 172.21.15.129:0/3712160252 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c032810 con 0x7f448c0c9030 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 -- 172.21.15.129:0/3712160252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c0c9030 msgr2=0x7f448c158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 --2- 172.21.15.129:0/3712160252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c0c9030 0x7f448c158970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f448c152f90 tx=0x7f447c02a1e0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 -- 172.21.15.129:0/3712160252 shutdown_connections 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 --2- 172.21.15.129:0/3712160252 >> v2:172.21.15.171:3300/0 conn(0x7f448c0c8b20 0x7f448c158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 --2- 172.21.15.129:0/3712160252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c0c9030 0x7f448c158970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 -- 172.21.15.129:0/3712160252 >> 172.21.15.129:0/3712160252 conn(0x7f448c0acbf0 msgr2=0x7f448c0ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 -- 172.21.15.129:0/3712160252 shutdown_connections 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.327+0000 7f449141c640 1 -- 172.21.15.129:0/3712160252 wait complete. 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.328+0000 7f449141c640 1 Processor -- start 2024-04-25T01:27:33.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.328+0000 7f449141c640 1 -- start start 2024-04-25T01:27:33.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.328+0000 7f449141c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f448c0c8b20 0x7f448c157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.328+0000 7f449141c640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f448c0c9030 0x7f448c14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.328+0000 7f449141c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c152bb0 0x7f448c14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f449141c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f448c156250 con 0x7f448c152bb0 2024-04-25T01:27:33.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f449141c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f448c155a50 con 0x7f448c0c8b20 2024-04-25T01:27:33.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f449141c640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f448c0c94f0 con 0x7f448c0c9030 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c152bb0 0x7f448c14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c152bb0 0x7f448c14c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55448/0 (socket says 172.21.15.129:55448) 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 -- 172.21.15.129:0/3738436381 learned_addr learned my addr 172.21.15.129:0/3738436381 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448affd640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f448c0c8b20 0x7f448c157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448a7fc640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f448c0c9030 0x7f448c14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 -- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f448c0c9030 msgr2=0x7f448c14bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f448c0c9030 0x7f448c14bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 -- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f448c0c8b20 msgr2=0x7f448c157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f448c0c8b20 0x7f448c157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 -- 172.21.15.129:0/3738436381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c1eb650 con 0x7f448c152bb0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f448b7fe640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c152bb0 0x7f448c14c160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4484007e40 tx=0x7f448400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484014620 con 0x7f448c152bb0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4484019070 con 0x7f448c152bb0 2024-04-25T01:27:33.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c1eb930 con 0x7f448c152bb0 2024-04-25T01:27:33.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484014a20 con 0x7f448c152bb0 2024-04-25T01:27:33.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.329+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c1ebef0 con 0x7f448c152bb0 2024-04-25T01:27:33.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.330+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f448c068fd0 con 0x7f448c152bb0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4484014bc0 con 0x7f448c152bb0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f44737fe640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f447806be30 0x7f447806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f448408c790 con 0x7f448c152bb0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f44737fe640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f4478075850 0x7f4478077cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f448affd640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f447806be30 0x7f447806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 --> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44780783c0 con 0x7f4478075850 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f448408cc20 con 0x7f448c152bb0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.331+0000 7f448a7fc640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f4478075850 0x7f4478077cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.332+0000 7f448affd640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f447806be30 0x7f447806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44800016e0 tx=0x7f4480009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.332+0000 7f448a7fc640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f4478075850 0x7f4478077cf0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.333+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== osd.4 v2:172.21.15.171:6808/4119213215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44780783c0 con 0x7f4478075850 2024-04-25T01:27:33.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.346+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 --> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f448c0ca8f0 con 0x7f4478075850 2024-04-25T01:27:33.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.346+0000 7f44737fe640 1 -- 172.21.15.129:0/3738436381 <== osd.4 v2:172.21.15.171:6808/4119213215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f448c0ca8f0 con 0x7f4478075850 2024-04-25T01:27:33.327 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:33.327 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:33.327 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f4478075850 msgr2=0x7f4478077cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6808/4119213215,v1:172.21.15.171:6810/4119213215] conn(0x7f4478075850 0x7f4478077cf0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f447806be30 msgr2=0x7f447806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f447806be30 0x7f447806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44800016e0 tx=0x7f4480009040 comp rx=0 tx=0).stop 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c152bb0 msgr2=0x7f448c14c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 --2- 172.21.15.129:0/3738436381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f448c152bb0 0x7f448c14c160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4484007e40 tx=0x7f448400e4b0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f448b7fe640 1 -- 172.21.15.129:0/3738436381 reap_dead start 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 shutdown_connections 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 >> 172.21.15.129:0/3738436381 conn(0x7f448c0acbf0 msgr2=0x7f448c1546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 shutdown_connections 2024-04-25T01:27:33.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.347+0000 7f449141c640 1 -- 172.21.15.129:0/3738436381 wait complete. 2024-04-25T01:27:33.337 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T01:27:33.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.456+0000 7fb3fb8bd640 1 Processor -- start 2024-04-25T01:27:33.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3fb8bd640 1 -- start start 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3fb8bd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb3f4152fa0 0x7fb3f40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3fb8bd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb3f40c5520 0x7fb3f41561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3fb8bd640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f40c9860 con 0x7fb3f40c7e50 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3fb8bd640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f4156720 con 0x7fb3f40c5520 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3fb8bd640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f40c5a70 con 0x7fb3f4152fa0 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3f8e58640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb3f40c5520 0x7fb3f41561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3f8e58640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fb3f40c5520 0x7fb3f41561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55450/0 (socket says 172.21.15.129:55450) 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3f8e58640 1 -- 172.21.15.129:0/2526140355 learned_addr learned my addr 172.21.15.129:0/2526140355 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.457+0000 7fb3f9659640 1 --2- 172.21.15.129:0/2526140355 >> v2:172.21.15.171:3300/0 conn(0x7fb3f4152fa0 0x7fb3f40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3f8e58640 1 -- 172.21.15.129:0/2526140355 >> v2:172.21.15.171:3300/0 conn(0x7fb3f4152fa0 msgr2=0x7fb3f40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:33.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3f8e58640 1 --2- 172.21.15.129:0/2526140355 >> v2:172.21.15.171:3300/0 conn(0x7fb3f4152fa0 0x7fb3f40ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3f8e58640 1 -- 172.21.15.129:0/2526140355 >> v1:172.21.15.171:6789/0 conn(0x7fb3f40c7e50 legacy=0x7fb3f40c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3ebfff640 1 -- 172.21.15.129:0/2526140355 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 30624913 0 0) 0x7fb3ec001490 con 0x7fb3f40c7e50 2024-04-25T01:27:33.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3f8e58640 1 -- 172.21.15.129:0/2526140355 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f40c5d50 con 0x7fb3f40c5520 2024-04-25T01:27:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3f8e58640 1 --2- 172.21.15.129:0/2526140355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c5520 0x7fb3f41561e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb3f408c590 tx=0x7fb3e402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57089b340fe9d8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.458+0000 7fb3ebfff640 1 -- 172.21.15.129:0/2526140355 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4037070 con 0x7fb3f40c5520 2024-04-25T01:27:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3ebfff640 1 -- 172.21.15.129:0/2526140355 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e4032440 con 0x7fb3f40c5520 2024-04-25T01:27:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3ebfff640 1 -- 172.21.15.129:0/2526140355 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4032810 con 0x7fb3f40c5520 2024-04-25T01:27:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/2526140355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c5520 msgr2=0x7fb3f41561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/2526140355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c5520 0x7fb3f41561e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb3f408c590 tx=0x7fb3e402a150 comp rx=0 tx=0).stop 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/2526140355 shutdown_connections 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/2526140355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c5520 0x7fb3f41561e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/2526140355 >> v2:172.21.15.171:3300/0 conn(0x7fb3f4152fa0 0x7fb3f40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/2526140355 >> 172.21.15.129:0/2526140355 conn(0x7fb3f406c400 msgr2=0x7fb3f406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/2526140355 shutdown_connections 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.459+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/2526140355 wait complete. 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.460+0000 7fb3fb8bd640 1 Processor -- start 2024-04-25T01:27:33.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.460+0000 7fb3fb8bd640 1 -- start start 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3fb8bd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb3f40c5520 0x7fb3f4157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3fb8bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 0x7fb3f41e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3fb8bd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3f4152fa0 0x7fb3f41e0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3fb8bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f40c93d0 con 0x7fb3f40c7e50 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3fb8bd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f40c6030 con 0x7fb3f4152fa0 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3fb8bd640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f40c9550 con 0x7fb3f40c5520 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 0x7fb3f41e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 0x7fb3f41e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55456/0 (socket says 172.21.15.129:55456) 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 -- 172.21.15.129:0/3399111536 learned_addr learned my addr 172.21.15.129:0/3399111536 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f9659640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb3f40c5520 0x7fb3f4157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f9e5a640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3f4152fa0 0x7fb3f41e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 -- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb3f40c5520 msgr2=0x7fb3f4157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb3f40c5520 0x7fb3f4157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 -- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3f4152fa0 msgr2=0x7fb3f41e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3f4152fa0 0x7fb3f41e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.461+0000 7fb3f8e58640 1 -- 172.21.15.129:0/3399111536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f40c1df0 con 0x7fb3f40c7e50 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.462+0000 7fb3f8e58640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 0x7fb3f41e03d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4000c00 tx=0x7fb3e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.462+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4037040 con 0x7fb3f40c7e50 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.462+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e4004250 con 0x7fb3f40c7e50 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.462+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f40c4620 con 0x7fb3f40c7e50 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.462+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4003610 con 0x7fb3f40c7e50 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.462+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f4158300 con 0x7fb3f40c7e50 2024-04-25T01:27:33.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.463+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f4075550 con 0x7fb3f40c7e50 2024-04-25T01:27:33.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.464+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3e40037b0 con 0x7fb3f40c7e50 2024-04-25T01:27:33.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.464+0000 7fb3e9ffb640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fb3d806be60 0x7fb3d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.464+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb3e40ac520 con 0x7fb3f40c7e50 2024-04-25T01:27:33.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.465+0000 7fb3f9659640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fb3d806be60 0x7fb3d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.465+0000 7fb3f9659640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fb3d806be60 0x7fb3d806e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009aa0 tx=0x7fb3f0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.472+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb3e407a5f0 con 0x7fb3f40c7e50 2024-04-25T01:27:33.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.653+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fb3f40ca0c0 con 0x7fb3d806be60 2024-04-25T01:27:33.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.653+0000 7fb3e9ffb640 1 -- 172.21.15.129:0/3399111536 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb3f40ca0c0 con 0x7fb3d806be60 2024-04-25T01:27:33.633 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fb3d806be60 msgr2=0x7fb3d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fb3d806be60 0x7fb3d806e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009aa0 tx=0x7fb3f0009190 comp rx=0 tx=0).stop 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 msgr2=0x7fb3f41e03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 0x7fb3f41e03d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4000c00 tx=0x7fb3e40046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 shutdown_connections 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fb3d806be60 0x7fb3d806e320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3f4152fa0 0x7fb3f41e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3f40c7e50 0x7fb3f41e03d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 --2- 172.21.15.129:0/3399111536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb3f40c5520 0x7fb3f4157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.655+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 >> 172.21.15.129:0/3399111536 conn(0x7fb3f406c400 msgr2=0x7fb3f40c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.656+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 shutdown_connections 2024-04-25T01:27:33.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.656+0000 7fb3fb8bd640 1 -- 172.21.15.129:0/3399111536 wait complete. 2024-04-25T01:27:33.648 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T01:27:33.648 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:33.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.773+0000 7f324355a640 1 Processor -- start 2024-04-25T01:27:33.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.773+0000 7f324355a640 1 -- start start 2024-04-25T01:27:33.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f324355a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c150830 0x7f323c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f324355a640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f323c0c6cf0 0x7f323c0c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f324355a640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f323c150c10 con 0x7f323c06ebe0 2024-04-25T01:27:33.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f324355a640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f323c0703c0 con 0x7f323c0c6cf0 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f324355a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f323c154190 con 0x7f323c150830 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f3240af5640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f323c06ebe0 0x7f323c0c8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60178/0 (socket says 172.21.15.129:60178) 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f3240af5640 1 -- 172.21.15.129:0/2371151612 learned_addr learned my addr 172.21.15.129:0/2371151612 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f3241af7640 1 --2- 172.21.15.129:0/2371151612 >> v2:172.21.15.171:3300/0 conn(0x7f323c0c6cf0 0x7f323c0c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f32412f6640 1 --2- 172.21.15.129:0/2371151612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c150830 0x7f323c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f32412f6640 1 -- 172.21.15.129:0/2371151612 >> v2:172.21.15.171:3300/0 conn(0x7f323c0c6cf0 msgr2=0x7f323c0c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f32412f6640 1 --2- 172.21.15.129:0/2371151612 >> v2:172.21.15.171:3300/0 conn(0x7f323c0c6cf0 0x7f323c0c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f32412f6640 1 -- 172.21.15.129:0/2371151612 >> v1:172.21.15.129:6789/0 conn(0x7f323c06ebe0 legacy=0x7f323c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f32412f6640 1 -- 172.21.15.129:0/2371151612 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c070660 con 0x7f323c150830 2024-04-25T01:27:33.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.774+0000 7f3241af7640 1 --2- 172.21.15.129:0/2371151612 >> v2:172.21.15.171:3300/0 conn(0x7f323c0c6cf0 0x7f323c0c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:33.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.775+0000 7f32412f6640 1 --2- 172.21.15.129:0/2371151612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c150830 0x7f323c153c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32380097a0 tx=0x7f323802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287334eca562fea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.776+0000 7f3233fff640 1 -- 172.21.15.129:0/2371151612 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238037070 con 0x7f323c150830 2024-04-25T01:27:33.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.776+0000 7f3233fff640 1 -- 172.21.15.129:0/2371151612 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323802ae50 con 0x7f323c150830 2024-04-25T01:27:33.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.776+0000 7f3233fff640 1 -- 172.21.15.129:0/2371151612 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238032660 con 0x7f323c150830 2024-04-25T01:27:33.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 -- 172.21.15.129:0/2371151612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c150830 msgr2=0x7f323c153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 --2- 172.21.15.129:0/2371151612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c150830 0x7f323c153c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32380097a0 tx=0x7f323802a100 comp rx=0 tx=0).stop 2024-04-25T01:27:33.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 -- 172.21.15.129:0/2371151612 shutdown_connections 2024-04-25T01:27:33.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 --2- 172.21.15.129:0/2371151612 >> v2:172.21.15.171:3300/0 conn(0x7f323c0c6cf0 0x7f323c0c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 --2- 172.21.15.129:0/2371151612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c150830 0x7f323c153c50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 -- 172.21.15.129:0/2371151612 >> 172.21.15.129:0/2371151612 conn(0x7f323c0aa760 msgr2=0x7f323c0aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 -- 172.21.15.129:0/2371151612 shutdown_connections 2024-04-25T01:27:33.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.777+0000 7f324355a640 1 -- 172.21.15.129:0/2371151612 wait complete. 2024-04-25T01:27:33.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.778+0000 7f324355a640 1 Processor -- start 2024-04-25T01:27:33.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.779+0000 7f324355a640 1 -- start start 2024-04-25T01:27:33.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323c06ebe0 0x7f323c1eb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c0c6cf0 0x7f323c0c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f323c150830 0x7f323c149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f323c070f00 con 0x7f323c06ebe0 2024-04-25T01:27:33.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f323c070c20 con 0x7f323c150830 2024-04-25T01:27:33.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f323c0c8710 con 0x7f323c0c6cf0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323c06ebe0 0x7f323c1eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323c06ebe0 0x7f323c1eb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55470/0 (socket says 172.21.15.129:55470) 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 -- 172.21.15.129:0/2534155063 learned_addr learned my addr 172.21.15.129:0/2534155063 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f3240af5640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c0c6cf0 0x7f323c0c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f3241af7640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f323c150830 0x7f323c149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 -- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c0c6cf0 msgr2=0x7f323c0c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f323c0c6cf0 0x7f323c0c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 -- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f323c150830 msgr2=0x7f323c149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f323c150830 0x7f323c149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 -- 172.21.15.129:0/2534155063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323c1ebd70 con 0x7f323c06ebe0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f32412f6640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323c06ebe0 0x7f323c1eb830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3238035040 tx=0x7f32380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238041070 con 0x7f323c06ebe0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323803c460 con 0x7f323c06ebe0 2024-04-25T01:27:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323c06cb90 con 0x7f323c06ebe0 2024-04-25T01:27:33.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.780+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238037040 con 0x7f323c06ebe0 2024-04-25T01:27:33.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.781+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323c06d150 con 0x7f323c06ebe0 2024-04-25T01:27:33.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.781+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f323c0681a0 con 0x7f323c06ebe0 2024-04-25T01:27:33.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.782+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f323803a070 con 0x7f323c06ebe0 2024-04-25T01:27:33.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3231ffb640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f322006be60 0x7f322006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f32380aba90 con 0x7f323c06ebe0 2024-04-25T01:27:33.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3240af5640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f322006be60 0x7f322006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3231ffb640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7f3220075870 0x7f3220077d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 --> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32380067f0 con 0x7f3220075870 2024-04-25T01:27:33.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32380abf20 con 0x7f323c06ebe0 2024-04-25T01:27:33.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3241af7640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7f3220075870 0x7f3220077d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3240af5640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f322006be60 0x7f322006e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f323c0aead0 tx=0x7f322c007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.783+0000 7f3241af7640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7f3220075870 0x7f3220077d10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.784+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== osd.5 v2:172.21.15.171:6809/486235959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32380067f0 con 0x7f3220075870 2024-04-25T01:27:33.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.797+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 --> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f323c0c7de0 con 0x7f3220075870 2024-04-25T01:27:33.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f3231ffb640 1 -- 172.21.15.129:0/2534155063 <== osd.5 v2:172.21.15.171:6809/486235959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f323c0c7de0 con 0x7f3220075870 2024-04-25T01:27:33.778 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:33.778 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:33.778 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7f3220075870 msgr2=0x7f3220077d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6809/486235959,v1:172.21.15.171:6811/486235959] conn(0x7f3220075870 0x7f3220077d10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f322006be60 msgr2=0x7f322006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f322006be60 0x7f322006e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f323c0aead0 tx=0x7f322c007e80 comp rx=0 tx=0).stop 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323c06ebe0 msgr2=0x7f323c1eb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 --2- 172.21.15.129:0/2534155063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f323c06ebe0 0x7f323c1eb830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3238035040 tx=0x7f32380046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f3241af7640 1 -- 172.21.15.129:0/2534155063 reap_dead start 2024-04-25T01:27:33.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 shutdown_connections 2024-04-25T01:27:33.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 >> 172.21.15.129:0/2534155063 conn(0x7f323c0aa760 msgr2=0x7f323c152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 shutdown_connections 2024-04-25T01:27:33.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.798+0000 7f324355a640 1 -- 172.21.15.129:0/2534155063 wait complete. 2024-04-25T01:27:33.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T01:27:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.907+0000 7fc5af815640 1 Processor -- start 2024-04-25T01:27:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.907+0000 7fc5af815640 1 -- start start 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5af815640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a80c5520 0x7fc5a80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5af815640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a80c9860 con 0x7fc5a81561e0 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5af815640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a8156690 con 0x7fc5a8152fa0 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5af815640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80c5a70 con 0x7fc5a80c5520 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5addb2640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc5a81561e0 0x7fc5a80c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60194/0 (socket says 172.21.15.129:60194) 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5addb2640 1 -- 172.21.15.129:0/3256091524 learned_addr learned my addr 172.21.15.129:0/3256091524 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.908+0000 7fc5acdb0640 1 --2- 172.21.15.129:0/3256091524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a80c5520 0x7fc5a80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.909+0000 7fc59ffff640 1 -- 172.21.15.129:0/3256091524 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315328338 0 0) 0x7fc5a0001490 con 0x7fc5a81561e0 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.909+0000 7fc59ffff640 1 -- 172.21.15.129:0/3256091524 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc594003520 con 0x7fc5a81561e0 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.909+0000 7fc59ffff640 1 -- 172.21.15.129:0/3256091524 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022359801 0 0) 0x7fc5a8156690 con 0x7fc5a8152fa0 2024-04-25T01:27:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.909+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3256091524 >> v1:172.21.15.171:6789/0 conn(0x7fc5a8152fa0 legacy=0x7fc5a80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.909+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3256091524 >> v1:172.21.15.129:6789/0 conn(0x7fc5a81561e0 legacy=0x7fc5a80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.890 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.909+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3256091524 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80c5d50 con 0x7fc5a80c5520 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.910+0000 7fc5acdb0640 1 --2- 172.21.15.129:0/3256091524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a80c5520 0x7fc5a80ca2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097d0 tx=0x7fc59802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b6e69cdee1584f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.910+0000 7fc59ffff640 1 -- 172.21.15.129:0/3256091524 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598037070 con 0x7fc5a80c5520 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.910+0000 7fc59ffff640 1 -- 172.21.15.129:0/3256091524 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59802ae40 con 0x7fc5a80c5520 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.910+0000 7fc59ffff640 1 -- 172.21.15.129:0/3256091524 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598032660 con 0x7fc5a80c5520 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.910+0000 7fc5af815640 1 -- 172.21.15.129:0/3256091524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a80c5520 msgr2=0x7fc5a80ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.910+0000 7fc5af815640 1 --2- 172.21.15.129:0/3256091524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a80c5520 0x7fc5a80ca2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097d0 tx=0x7fc59802a0f0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.911+0000 7fc5af815640 1 -- 172.21.15.129:0/3256091524 shutdown_connections 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.911+0000 7fc5af815640 1 --2- 172.21.15.129:0/3256091524 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a80c5520 0x7fc5a80ca2e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.911+0000 7fc5af815640 1 -- 172.21.15.129:0/3256091524 >> 172.21.15.129:0/3256091524 conn(0x7fc5a806c400 msgr2=0x7fc5a806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:33.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.911+0000 7fc5af815640 1 -- 172.21.15.129:0/3256091524 shutdown_connections 2024-04-25T01:27:33.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.911+0000 7fc5af815640 1 -- 172.21.15.129:0/3256091524 wait complete. 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.912+0000 7fc5af815640 1 Processor -- start 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.912+0000 7fc5af815640 1 -- start start 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5af815640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 0x7fc5a80be550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5af815640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 0x7fc5a81eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5af815640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5a81561e0 0x7fc5a80c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5af815640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80c93d0 con 0x7fc5a81561e0 2024-04-25T01:27:33.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5af815640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80c6030 con 0x7fc5a80c5520 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5af815640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80c9550 con 0x7fc5a8152fa0 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5addb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5a81561e0 0x7fc5a80c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 0x7fc5a81eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5ad5b1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 0x7fc5a80be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 0x7fc5a81eeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:52164/0 (socket says 172.21.15.129:52164) 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5ad5b1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 0x7fc5a80be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:51182/0 (socket says 172.21.15.129:51182) 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3290983352 learned_addr learned my addr 172.21.15.129:0/3290983352 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5addb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5a81561e0 0x7fc5a80c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55482/0 (socket says 172.21.15.129:55482) 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 msgr2=0x7fc5a80be550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 0x7fc5a80be550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3290983352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5a81561e0 msgr2=0x7fc5a80c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5a81561e0 0x7fc5a80c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 -- 172.21.15.129:0/3290983352 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80c4a20 con 0x7fc5a8152fa0 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5ad5b1640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 0x7fc5a80be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc5acdb0640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 0x7fc5a81eeb20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097d0 tx=0x7fc5980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598037040 con 0x7fc5a8152fa0 2024-04-25T01:27:33.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc598004030 con 0x7fc5a8152fa0 2024-04-25T01:27:33.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.913+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598004200 con 0x7fc5a8152fa0 2024-04-25T01:27:33.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.914+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a80c4d00 con 0x7fc5a8152fa0 2024-04-25T01:27:33.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.914+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81ec7b0 con 0x7fc5a8152fa0 2024-04-25T01:27:33.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.915+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a80c28b0 con 0x7fc5a8152fa0 2024-04-25T01:27:33.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.915+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc59803a070 con 0x7fc5a8152fa0 2024-04-25T01:27:33.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.916+0000 7fc59dffb640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc58c06be30 0x7fc58c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:33.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.916+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc5980ab690 con 0x7fc5a8152fa0 2024-04-25T01:27:33.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.916+0000 7fc5ad5b1640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc58c06be30 0x7fc58c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:33.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.917+0000 7fc5ad5b1640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc58c06be30 0x7fc58c06e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4000f60 tx=0x7fc5a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:33.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:33.921+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc598041070 con 0x7fc5a8152fa0 2024-04-25T01:27:34.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.121+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fc5a81ef7a0 con 0x7fc58c06be30 2024-04-25T01:27:34.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.122+0000 7fc59dffb640 1 -- 172.21.15.129:0/3290983352 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5a81ef7a0 con 0x7fc58c06be30 2024-04-25T01:27:34.102 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T01:27:34.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc58c06be30 msgr2=0x7fc58c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc58c06be30 0x7fc58c06e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4000f60 tx=0x7fc5a4009040 comp rx=0 tx=0).stop 2024-04-25T01:27:34.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 msgr2=0x7fc5a81eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 0x7fc5a81eeb20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097d0 tx=0x7fc5980046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 shutdown_connections 2024-04-25T01:27:34.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc58c06be30 0x7fc58c06e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a8152fa0 0x7fc5a81eeb20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5a81561e0 0x7fc5a80c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 --2- 172.21.15.129:0/3290983352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a80c5520 0x7fc5a80be550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 >> 172.21.15.129:0/3290983352 conn(0x7fc5a806c400 msgr2=0x7fc5a8154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 shutdown_connections 2024-04-25T01:27:34.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.124+0000 7fc5af815640 1 -- 172.21.15.129:0/3290983352 wait complete. 2024-04-25T01:27:34.115 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T01:27:34.115 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:34.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.238+0000 7f71ecf63640 1 Processor -- start 2024-04-25T01:27:34.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.238+0000 7f71ecf63640 1 -- start start 2024-04-25T01:27:34.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71ecf63640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71e8158970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71ecf63640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e806cfe0 con 0x7f71e8152bb0 2024-04-25T01:27:34.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71ecf63640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e8155490 con 0x7f71e80c8b20 2024-04-25T01:27:34.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71ecf63640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f71e8155dc0 con 0x7f71e80c9030 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71e659c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f71e8152bb0 0x7f71e80c55a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60206/0 (socket says 172.21.15.129:60206) 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71e659c640 1 -- 172.21.15.129:0/2145052802 learned_addr learned my addr 172.21.15.129:0/2145052802 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71e8158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805540224 0 0) 0x7f71e806cfe0 con 0x7f71e8152bb0 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.239+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71d0003520 con 0x7f71e8152bb0 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949629819 0 0) 0x7f71e8155490 con 0x7f71e80c8b20 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e806cfe0 con 0x7f71e80c8b20 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3675069642 0 0) 0x7f71d0003520 con 0x7f71e8152bb0 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 msgr2=0x7f71e8158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71e8158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 >> v1:172.21.15.171:6789/0 conn(0x7f71e80c8b20 legacy=0x7f71e815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e8155770 con 0x7f71e8152bb0 2024-04-25T01:27:34.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f71e0001670 con 0x7f71e8152bb0 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71e8158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 >> v1:172.21.15.129:6789/0 conn(0x7f71e8152bb0 legacy=0x7f71e80c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71d0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71d00040b0 0x7f71d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71d0004d40 0x7f71d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8155dc0 con 0x7f71d00040b0 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71d000bb00 con 0x7f71d0004d40 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f71d000bd00 con 0x7f71e80c9030 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71d00040b0 0x7f71d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e659c640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71d0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e6d9d640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71d0004d40 0x7f71d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 -- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 msgr2=0x7f71d0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71d0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 -- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71d0004d40 msgr2=0x7f71d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71d0004d40 0x7f71d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e5d9b640 1 -- 172.21.15.129:0/2145052802 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d0003520 con 0x7f71d00040b0 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e6d9d640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71d0004d40 0x7f71d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.240+0000 7f71e659c640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71e80c9030 0x7f71d0003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.241+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71d00040b0 0x7f71d0004800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009c70 tx=0x7f71d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8471d7f08bd3e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.241+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc036070 con 0x7f71d00040b0 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.241+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc031400 con 0x7f71d00040b0 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.241+0000 7f71e559a640 1 -- 172.21.15.129:0/2145052802 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc009840 con 0x7f71d00040b0 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71ecf63640 1 -- 172.21.15.129:0/2145052802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71d00040b0 msgr2=0x7f71d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71ecf63640 1 --2- 172.21.15.129:0/2145052802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71d00040b0 0x7f71d0004800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009c70 tx=0x7f71d0005830 comp rx=0 tx=0).stop 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71e6d9d640 1 -- 172.21.15.129:0/2145052802 reap_dead start 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71ecf63640 1 -- 172.21.15.129:0/2145052802 shutdown_connections 2024-04-25T01:27:34.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71ecf63640 1 -- 172.21.15.129:0/2145052802 >> 172.21.15.129:0/2145052802 conn(0x7f71e80acbf0 msgr2=0x7f71e80ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71ecf63640 1 -- 172.21.15.129:0/2145052802 shutdown_connections 2024-04-25T01:27:34.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.242+0000 7f71ecf63640 1 -- 172.21.15.129:0/2145052802 wait complete. 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.243+0000 7f71ecf63640 1 Processor -- start 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.243+0000 7f71ecf63640 1 -- start start 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71d00040b0 0x7f71e8151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e81ee2b0 0x7f71e80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e80c4970 0x7f71e80b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e80c94f0 con 0x7f71e81ee2b0 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8156250 con 0x7f71e80c4970 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f71e806d3d0 con 0x7f71d00040b0 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e81ee2b0 0x7f71e80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e81ee2b0 0x7f71e80afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55496/0 (socket says 172.21.15.129:55496) 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 -- 172.21.15.129:0/673779113 learned_addr learned my addr 172.21.15.129:0/673779113 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:34.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e659c640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71d00040b0 0x7f71e8151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e6d9d640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e80c4970 0x7f71e80b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 -- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71d00040b0 msgr2=0x7f71e8151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71d00040b0 0x7f71e8151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 -- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e80c4970 msgr2=0x7f71e80b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71e80c4970 0x7f71e80b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 -- 172.21.15.129:0/673779113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e80af0d0 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71e5d9b640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e81ee2b0 0x7f71e80afd30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009d90 tx=0x7f71dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc040020 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc0093e0 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e80c1d60 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.244+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc036040 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.245+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e8151bd0 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.245+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71e8068fd0 con 0x7f71e81ee2b0 2024-04-25T01:27:34.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.246+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71dc043020 con 0x7f71e81ee2b0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.246+0000 7f71d6ffd640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.246+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f71dc0ab740 con 0x7f71e81ee2b0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.246+0000 7f71e659c640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.246+0000 7f71d6ffd640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7f71c8075870 0x7f71c8077d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.246+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 --> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71dc031400 con 0x7f71c8075870 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.247+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71dc0b2050 con 0x7f71e81ee2b0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.247+0000 7f71e6d9d640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7f71c8075870 0x7f71c8077d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.247+0000 7f71e659c640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f71c806be60 0x7f71c806e320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71e0001200 tx=0x7f71e0007c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.247+0000 7f71e6d9d640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7f71c8075870 0x7f71c8077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.248+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== osd.6 v2:172.21.15.171:6824/2873795804 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71dc031400 con 0x7f71c8075870 2024-04-25T01:27:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.261+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 --> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f71e806d1b0 con 0x7f71c8075870 2024-04-25T01:27:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.261+0000 7f71d6ffd640 1 -- 172.21.15.129:0/673779113 <== osd.6 v2:172.21.15.171:6824/2873795804 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71e806d1b0 con 0x7f71c8075870 2024-04-25T01:27:34.241 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:34.242 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:34.242 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.261+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7f71c8075870 msgr2=0x7f71c8077d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6824/2873795804,v1:172.21.15.171:6825/2873795804] conn(0x7f71c8075870 0x7f71c8077d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f71c806be60 msgr2=0x7f71c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f71c806be60 0x7f71c806e320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71e0001200 tx=0x7f71e0007c50 comp rx=0 tx=0).stop 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e81ee2b0 msgr2=0x7f71e80afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 --2- 172.21.15.129:0/673779113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e81ee2b0 0x7f71e80afd30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009d90 tx=0x7f71dc004730 comp rx=0 tx=0).stop 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71e6d9d640 1 -- 172.21.15.129:0/673779113 reap_dead start 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 shutdown_connections 2024-04-25T01:27:34.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 >> 172.21.15.129:0/673779113 conn(0x7f71e80acbf0 msgr2=0x7f71e8153670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 shutdown_connections 2024-04-25T01:27:34.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.262+0000 7f71ecf63640 1 -- 172.21.15.129:0/673779113 wait complete. 2024-04-25T01:27:34.252 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T01:27:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.371+0000 7f75509b7640 1 Processor -- start 2024-04-25T01:27:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.371+0000 7f75509b7640 1 -- start start 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f75509b7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f754c0c5520 0x7f754c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f75509b7640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f754c1561e0 0x7f754c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f75509b7640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c0c9860 con 0x7f754c152fa0 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f75509b7640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0c5a70 con 0x7f754c0c5520 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f75509b7640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f754c0c5d50 con 0x7f754c1561e0 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f7549d9b640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f754c0c5520 0x7f754c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f754a59c640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f754c152fa0 0x7f754c0c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60222/0 (socket says 172.21.15.129:60222) 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f754a59c640 1 -- 172.21.15.129:0/3013339176 learned_addr learned my addr 172.21.15.129:0/3013339176 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.372+0000 7f754ad9d640 1 --2- 172.21.15.129:0/3013339176 >> v2:172.21.15.171:3300/0 conn(0x7f754c1561e0 0x7f754c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.373+0000 7f7549d9b640 1 -- 172.21.15.129:0/3013339176 >> v2:172.21.15.171:3300/0 conn(0x7f754c1561e0 msgr2=0x7f754c0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.373+0000 7f7549d9b640 1 --2- 172.21.15.129:0/3013339176 >> v2:172.21.15.171:3300/0 conn(0x7f754c1561e0 0x7f754c0c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.373+0000 7f7549d9b640 1 -- 172.21.15.129:0/3013339176 >> v1:172.21.15.129:6789/0 conn(0x7f754c152fa0 legacy=0x7f754c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.373+0000 7f7549d9b640 1 -- 172.21.15.129:0/3013339176 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0c7650 con 0x7f754c0c5520 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.373+0000 7f7549d9b640 1 --2- 172.21.15.129:0/3013339176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0ca2e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f754c0aa580 tx=0x7f753c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b933e5c4179681d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.373+0000 7f754959a640 1 -- 172.21.15.129:0/3013339176 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c037070 con 0x7f754c0c5520 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f754959a640 1 -- 172.21.15.129:0/3013339176 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753c02add0 con 0x7f754c0c5520 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f754959a640 1 -- 172.21.15.129:0/3013339176 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c032660 con 0x7f754c0c5520 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 -- 172.21.15.129:0/3013339176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 msgr2=0x7f754c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 --2- 172.21.15.129:0/3013339176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0ca2e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f754c0aa580 tx=0x7f753c02a080 comp rx=0 tx=0).stop 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 -- 172.21.15.129:0/3013339176 shutdown_connections 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 --2- 172.21.15.129:0/3013339176 >> v2:172.21.15.171:3300/0 conn(0x7f754c1561e0 0x7f754c0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 --2- 172.21.15.129:0/3013339176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0ca2e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 -- 172.21.15.129:0/3013339176 >> 172.21.15.129:0/3013339176 conn(0x7f754c06c400 msgr2=0x7f754c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 -- 172.21.15.129:0/3013339176 shutdown_connections 2024-04-25T01:27:34.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.374+0000 7f75509b7640 1 -- 172.21.15.129:0/3013339176 wait complete. 2024-04-25T01:27:34.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.375+0000 7f75509b7640 1 Processor -- start 2024-04-25T01:27:34.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.375+0000 7f75509b7640 1 -- start start 2024-04-25T01:27:34.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f75509b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f75509b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f754c152fa0 0x7f754c0c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f75509b7640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f754c1561e0 0x7f754c0c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f75509b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0c9200 con 0x7f754c0c5520 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f75509b7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f754c0c6030 con 0x7f754c152fa0 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f75509b7640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f754c0c9380 con 0x7f754c1561e0 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f754a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f754a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55512/0 (socket says 172.21.15.129:55512) 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f754a59c640 1 -- 172.21.15.129:0/2976243612 learned_addr learned my addr 172.21.15.129:0/2976243612 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f754ad9d640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f754c1561e0 0x7f754c0c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f7549d9b640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f754c152fa0 0x7f754c0c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.376+0000 7f754a59c640 1 -- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f754c1561e0 msgr2=0x7f754c0c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f754a59c640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f754c1561e0 0x7f754c0c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f754a59c640 1 -- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f754c152fa0 msgr2=0x7f754c0c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f754a59c640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f754c152fa0 0x7f754c0c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f754a59c640 1 -- 172.21.15.129:0/2976243612 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c159470 con 0x7f754c0c5520 2024-04-25T01:27:34.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f754a59c640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0c3060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f754000c400 tx=0x7f754000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540017070 con 0x7f754c0c5520 2024-04-25T01:27:34.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7540005ce0 con 0x7f754c0c5520 2024-04-25T01:27:34.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75400072f0 con 0x7f754c0c5520 2024-04-25T01:27:34.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c159750 con 0x7f754c0c5520 2024-04-25T01:27:34.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.377+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c0bee20 con 0x7f754c0c5520 2024-04-25T01:27:34.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.378+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c075550 con 0x7f754c0c5520 2024-04-25T01:27:34.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.378+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f754001a070 con 0x7f754c0c5520 2024-04-25T01:27:34.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.379+0000 7f753affd640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f752c06be30 0x7f752c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.379+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f754008a550 con 0x7f754c0c5520 2024-04-25T01:27:34.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.379+0000 7f7549d9b640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f752c06be30 0x7f752c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.379+0000 7f7549d9b640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f752c06be30 0x7f752c06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f753c034000 tx=0x7f753c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.383+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f754c0c9860 con 0x7f754c0c5520 2024-04-25T01:27:34.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.562+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f754c159220 con 0x7f752c06be30 2024-04-25T01:27:34.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.563+0000 7f753affd640 1 -- 172.21.15.129:0/2976243612 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f754c159220 con 0x7f752c06be30 2024-04-25T01:27:34.543 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f752c06be30 msgr2=0x7f752c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f752c06be30 0x7f752c06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f753c034000 tx=0x7f753c02e000 comp rx=0 tx=0).stop 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 msgr2=0x7f754c0c3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0c3060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f754000c400 tx=0x7f754000c8c0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 shutdown_connections 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f752c06be30 0x7f752c06e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f754c1561e0 0x7f754c0c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f754c152fa0 0x7f754c0c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 --2- 172.21.15.129:0/2976243612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c0c5520 0x7f754c0c3060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 >> 172.21.15.129:0/2976243612 conn(0x7f754c06c400 msgr2=0x7f754c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 shutdown_connections 2024-04-25T01:27:34.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.565+0000 7f75509b7640 1 -- 172.21.15.129:0/2976243612 wait complete. 2024-04-25T01:27:34.555 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T01:27:34.556 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.678+0000 7f7e7fbf9640 1 Processor -- start 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.678+0000 7f7e7fbf9640 1 -- start start 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7fbf9640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7e780c4090 0x7f7e780c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7fbf9640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7e780c1ca0 0x7f7e780c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7fbf9640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e780c6100 con 0x7f7e780c3590 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7fbf9640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f7e780c5a00 con 0x7f7e780c1ca0 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7fbf9640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7808b350 con 0x7f7e780c4090 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7e780c1ca0 0x7f7e780c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7d194640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7e780c4090 0x7f7e780c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f7e780c1ca0 0x7f7e780c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55516/0 (socket says 172.21.15.129:55516) 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 -- 172.21.15.129:0/1838682447 learned_addr learned my addr 172.21.15.129:0/1838682447 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7d995640 1 --1- 172.21.15.129:0/1838682447 >> v1:172.21.15.171:6789/0 conn(0x7f7e780c3590 0x7f7e780ac160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:44340/0 (socket says 172.21.15.129:44340) 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 -- 172.21.15.129:0/1838682447 >> v2:172.21.15.171:3300/0 conn(0x7f7e780c4090 msgr2=0x7f7e780c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 --2- 172.21.15.129:0/1838682447 >> v2:172.21.15.171:3300/0 conn(0x7f7e780c4090 0x7f7e780c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 -- 172.21.15.129:0/1838682447 >> v1:172.21.15.171:6789/0 conn(0x7f7e780c3590 legacy=0x7f7e780ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.679+0000 7f7e7e196640 1 -- 172.21.15.129:0/1838682447 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e780c63e0 con 0x7f7e780c1ca0 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.680+0000 7f7e7e196640 1 --2- 172.21.15.129:0/1838682447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e780c5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e780c67f0 tx=0x7f7e7002abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88d90ea4a17b460 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.680+0000 7f7e7c993640 1 -- 172.21.15.129:0/1838682447 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7002c920 con 0x7f7e780c1ca0 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.680+0000 7f7e7c993640 1 -- 172.21.15.129:0/1838682447 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7002cac0 con 0x7f7e780c1ca0 2024-04-25T01:27:34.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.680+0000 7f7e7c993640 1 -- 172.21.15.129:0/1838682447 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7002b440 con 0x7f7e780c1ca0 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.680+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/1838682447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 msgr2=0x7f7e780c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 --2- 172.21.15.129:0/1838682447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e780c5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e780c67f0 tx=0x7f7e7002abc0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/1838682447 shutdown_connections 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 --2- 172.21.15.129:0/1838682447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e780c5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 --2- 172.21.15.129:0/1838682447 >> v2:172.21.15.171:3300/0 conn(0x7f7e780c4090 0x7f7e780c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/1838682447 >> 172.21.15.129:0/1838682447 conn(0x7f7e780c6890 msgr2=0x7f7e780c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/1838682447 shutdown_connections 2024-04-25T01:27:34.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.681+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/1838682447 wait complete. 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 Processor -- start 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 -- start start 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e781ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e780c3590 0x7f7e781eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e78073320 0x7f7e781fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e780a5f70 con 0x7f7e780c1ca0 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7808b5a0 con 0x7f7e780c3590 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.682+0000 7f7e7fbf9640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f7e780a60f0 con 0x7f7e78073320 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e781ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e781ec9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55518/0 (socket says 172.21.15.129:55518) 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 -- 172.21.15.129:0/2627948515 learned_addr learned my addr 172.21.15.129:0/2627948515 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d194640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e780c3590 0x7f7e781eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7e196640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e78073320 0x7f7e781fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 -- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e78073320 msgr2=0x7f7e781fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e78073320 0x7f7e781fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 -- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e780c3590 msgr2=0x7f7e781eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e780c3590 0x7f7e781eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 -- 172.21.15.129:0/2627948515 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7806e0c0 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d194640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e780c3590 0x7f7e781eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7d995640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e781ec9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400c430 tx=0x7f7e7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e740074e0 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e74007680 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74004a00 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e781ec190 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.683+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e781fa4e0 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.684+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e780c5270 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e7401a020 con 0x7f7e780c1ca0 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e6a7fc640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e5c06be30 0x7f7e5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7e7408a930 con 0x7f7e780c1ca0 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e7d194640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e5c06be30 0x7f7e5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e6a7fc640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7f7e5c075840 0x7f7e5c077ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 --> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e5c0783b0 con 0x7f7e5c075840 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.685+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e7408f2c0 con 0x7f7e780c1ca0 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.686+0000 7f7e7e196640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7f7e5c075840 0x7f7e5c077ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.686+0000 7f7e7d194640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e5c06be30 0x7f7e5c06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a9e70 tx=0x7f7e6c00c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.686+0000 7f7e7e196640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7f7e5c075840 0x7f7e5c077ce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.687+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== osd.7 v2:172.21.15.171:6800/3134257063 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e5c0783b0 con 0x7f7e5c075840 2024-04-25T01:27:34.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 --> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7e78068fd0 con 0x7f7e5c075840 2024-04-25T01:27:34.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e6a7fc640 1 -- 172.21.15.129:0/2627948515 <== osd.7 v2:172.21.15.171:6800/3134257063 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7e78068fd0 con 0x7f7e5c075840 2024-04-25T01:27:34.679 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-25T01:27:34.679 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:27:34.679 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-25T01:27:34.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7f7e5c075840 msgr2=0x7f7e5c077ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6800/3134257063,v1:172.21.15.171:6801/3134257063] conn(0x7f7e5c075840 0x7f7e5c077ce0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e5c06be30 msgr2=0x7f7e5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f7e5c06be30 0x7f7e5c06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a9e70 tx=0x7f7e6c00c040 comp rx=0 tx=0).stop 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 msgr2=0x7f7e781ec9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 --2- 172.21.15.129:0/2627948515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e780c1ca0 0x7f7e781ec9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400c430 tx=0x7f7e7400c8f0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7e196640 1 -- 172.21.15.129:0/2627948515 reap_dead start 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 shutdown_connections 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 >> 172.21.15.129:0/2627948515 conn(0x7f7e780c6890 msgr2=0x7f7e780a86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.700+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 shutdown_connections 2024-04-25T01:27:34.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.701+0000 7f7e7fbf9640 1 -- 172.21.15.129:0/2627948515 wait complete. 2024-04-25T01:27:34.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T01:27:34.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.810+0000 7f40fe3ab640 1 Processor -- start 2024-04-25T01:27:34.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.811+0000 7f40fe3ab640 1 -- start start 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.811+0000 7f40fe3ab640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f40f80ca2e0 0x7f40f81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.811+0000 7f40fe3ab640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f80c9860 con 0x7f40f8152fa0 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.811+0000 7f40fe3ab640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f80ca730 con 0x7f40f80c5520 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.811+0000 7f40fe3ab640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f40f8156720 con 0x7f40f80ca2e0 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40fc948640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f40f80ca2e0 0x7f40f81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40f7fff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f40f8152fa0 0x7f40f80c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60238/0 (socket says 172.21.15.129:60238) 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40f7fff640 1 -- 172.21.15.129:0/397830130 learned_addr learned my addr 172.21.15.129:0/397830130 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40f6ffd640 1 -- 172.21.15.129:0/397830130 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117940809 0 0) 0x7f40ec001110 con 0x7f40f8152fa0 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40f6ffd640 1 -- 172.21.15.129:0/397830130 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40e0003520 con 0x7f40f8152fa0 2024-04-25T01:27:34.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40f6ffd640 1 -- 172.21.15.129:0/397830130 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2255394683 0 0) 0x7f40f80ca730 con 0x7f40f80c5520 2024-04-25T01:27:34.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40fc948640 1 -- 172.21.15.129:0/397830130 >> v1:172.21.15.171:6789/0 conn(0x7f40f80c5520 legacy=0x7f40f80c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40fc948640 1 -- 172.21.15.129:0/397830130 >> v1:172.21.15.129:6789/0 conn(0x7f40f8152fa0 legacy=0x7f40f80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.812+0000 7f40fc948640 1 -- 172.21.15.129:0/397830130 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f80c5a70 con 0x7f40f80ca2e0 2024-04-25T01:27:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fc948640 1 --2- 172.21.15.129:0/397830130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81561e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d90 tx=0x7f40f002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a8bf76c4c29ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40f6ffd640 1 -- 172.21.15.129:0/397830130 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0031600 con 0x7f40f80ca2e0 2024-04-25T01:27:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40f6ffd640 1 -- 172.21.15.129:0/397830130 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f00317a0 con 0x7f40f80ca2e0 2024-04-25T01:27:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40f6ffd640 1 -- 172.21.15.129:0/397830130 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0031b70 con 0x7f40f80ca2e0 2024-04-25T01:27:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 -- 172.21.15.129:0/397830130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 msgr2=0x7f40f81561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/397830130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81561e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d90 tx=0x7f40f002a5d0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 -- 172.21.15.129:0/397830130 shutdown_connections 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/397830130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81561e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 -- 172.21.15.129:0/397830130 >> 172.21.15.129:0/397830130 conn(0x7f40f806c400 msgr2=0x7f40f806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 -- 172.21.15.129:0/397830130 shutdown_connections 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.813+0000 7f40fe3ab640 1 -- 172.21.15.129:0/397830130 wait complete. 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.814+0000 7f40fe3ab640 1 Processor -- start 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.814+0000 7f40fe3ab640 1 -- start start 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fe3ab640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40f80c5520 0x7f40f80c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fe3ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fe3ab640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40f8152fa0 0x7f40f80bef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fe3ab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80c6030 con 0x7f40f80ca2e0 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fe3ab640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80c7650 con 0x7f40f80c5520 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fe3ab640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f40f80c93d0 con 0x7f40f8152fa0 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81ee7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55534/0 (socket says 172.21.15.129:55534) 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 -- 172.21.15.129:0/1848302290 learned_addr learned my addr 172.21.15.129:0/1848302290 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f7fff640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40f80c5520 0x7f40f80c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40fc948640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40f8152fa0 0x7f40f80bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 -- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40f8152fa0 msgr2=0x7f40f80bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40f8152fa0 0x7f40f80bef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 -- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40f80c5520 msgr2=0x7f40f80c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40f80c5520 0x7f40f80c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.815+0000 7f40f77fe640 1 -- 172.21.15.129:0/1848302290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f814f190 con 0x7f40f80ca2e0 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.816+0000 7f40f77fe640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81ee7c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c410 tx=0x7f40e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.816+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8017070 con 0x7f40f80ca2e0 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.816+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e80035f0 con 0x7f40f80ca2e0 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.816+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f81ef240 con 0x7f40f80ca2e0 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.816+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80039e0 con 0x7f40f80ca2e0 2024-04-25T01:27:34.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.816+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f80c0b60 con 0x7f40f80ca2e0 2024-04-25T01:27:34.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.817+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f8075550 con 0x7f40f80ca2e0 2024-04-25T01:27:34.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.817+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40e801a070 con 0x7f40f80ca2e0 2024-04-25T01:27:34.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.818+0000 7f40f4ff9640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f40e406be60 0x7f40e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:34.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.818+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f40e808a500 con 0x7f40f80ca2e0 2024-04-25T01:27:34.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.818+0000 7f40f7fff640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f40e406be60 0x7f40e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:34.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.818+0000 7f40f7fff640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f40e406be60 0x7f40e406e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005880 tx=0x7f40ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:34.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:34.822+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40e80584f0 con 0x7f40f80ca2e0 2024-04-25T01:27:34.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.013+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f40f814ef40 con 0x7f40e406be60 2024-04-25T01:27:34.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.013+0000 7f40f4ff9640 1 -- 172.21.15.129:0/1848302290 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f40f814ef40 con 0x7f40e406be60 2024-04-25T01:27:34.992 INFO:teuthology.orchestra.run.smithi129.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T01:27:34.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f40e406be60 msgr2=0x7f40e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f40e406be60 0x7f40e406e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005880 tx=0x7f40ec009040 comp rx=0 tx=0).stop 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 msgr2=0x7f40f81ee7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81ee7c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c410 tx=0x7f40e800c8d0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 shutdown_connections 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f40e406be60 0x7f40e406e320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40f8152fa0 0x7f40f80bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40f80ca2e0 0x7f40f81ee7c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 --2- 172.21.15.129:0/1848302290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40f80c5520 0x7f40f80c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 >> 172.21.15.129:0/1848302290 conn(0x7f40f806c400 msgr2=0x7f40f806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 shutdown_connections 2024-04-25T01:27:34.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.015+0000 7f40fe3ab640 1 -- 172.21.15.129:0/1848302290 wait complete. 2024-04-25T01:27:35.006 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:27:35.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.131+0000 7fc78da15640 1 Processor -- start 2024-04-25T01:27:35.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.131+0000 7fc78da15640 1 -- start start 2024-04-25T01:27:35.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc78da15640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc78806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc78da15640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78806e950 con 0x7fc78806c2f0 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc78da15640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc788153260 con 0x7fc78806e4c0 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc78da15640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc788154020 con 0x7fc788072580 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc786ffd640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7fc78806c2f0 0x7fc788072c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:60248/0 (socket says 172.21.15.129:60248) 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc786ffd640 1 -- 172.21.15.129:0/4013209458 learned_addr learned my addr 172.21.15.129:0/4013209458 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc7867fc640 1 --2- 172.21.15.129:0/4013209458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc78806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc785ffb640 1 -- 172.21.15.129:0/4013209458 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720629663 0 0) 0x7fc78806e950 con 0x7fc78806c2f0 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc785ffb640 1 -- 172.21.15.129:0/4013209458 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc770003520 con 0x7fc78806c2f0 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc7867fc640 1 -- 172.21.15.129:0/4013209458 >> v1:172.21.15.171:6789/0 conn(0x7fc78806e4c0 legacy=0x7fc7880c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc785ffb640 1 -- 172.21.15.129:0/4013209458 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269156560 0 0) 0x7fc788153260 con 0x7fc78806e4c0 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc7867fc640 1 -- 172.21.15.129:0/4013209458 >> v1:172.21.15.129:6789/0 conn(0x7fc78806c2f0 legacy=0x7fc788072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.132+0000 7fc7867fc640 1 -- 172.21.15.129:0/4013209458 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc788153540 con 0x7fc788072580 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.133+0000 7fc7867fc640 1 --2- 172.21.15.129:0/4013209458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc78806df80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0097d0 tx=0x7fc77c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93f9dba5a1d76be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc785ffb640 1 -- 172.21.15.129:0/4013209458 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c037070 con 0x7fc788072580 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc785ffb640 1 -- 172.21.15.129:0/4013209458 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c032440 con 0x7fc788072580 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc785ffb640 1 -- 172.21.15.129:0/4013209458 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c032810 con 0x7fc788072580 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 -- 172.21.15.129:0/4013209458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 msgr2=0x7fc78806df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 --2- 172.21.15.129:0/4013209458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc78806df80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0097d0 tx=0x7fc77c02a210 comp rx=0 tx=0).stop 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 -- 172.21.15.129:0/4013209458 shutdown_connections 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 --2- 172.21.15.129:0/4013209458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc78806df80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 -- 172.21.15.129:0/4013209458 >> 172.21.15.129:0/4013209458 conn(0x7fc7880c3c80 msgr2=0x7fc7880c4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 -- 172.21.15.129:0/4013209458 shutdown_connections 2024-04-25T01:27:35.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.134+0000 7fc78da15640 1 -- 172.21.15.129:0/4013209458 wait complete. 2024-04-25T01:27:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 Processor -- start 2024-04-25T01:27:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 -- start start 2024-04-25T01:27:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 0x7fc7881ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc78806e4c0 0x7fc788155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc7881558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc788153b00 con 0x7fc78806c2f0 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc7880734c0 con 0x7fc78806e4c0 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.136+0000 7fc78da15640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc78806c6d0 con 0x7fc788072580 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc786ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 0x7fc7881ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc786ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 0x7fc7881ebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55542/0 (socket says 172.21.15.129:55542) 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc7881558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc786ffd640 1 -- 172.21.15.129:0/3132232705 learned_addr learned my addr 172.21.15.129:0/3132232705 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7867fc640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc78806e4c0 0x7fc788155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 -- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc78806e4c0 msgr2=0x7fc788155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc78806e4c0 0x7fc788155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 -- 172.21.15.129:0/3132232705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 msgr2=0x7fc7881ebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 0x7fc7881ebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 -- 172.21.15.129:0/3132232705 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc788155de0 con 0x7fc788072580 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc786ffd640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 0x7fc7881ebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7867fc640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc78806e4c0 0x7fc788155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc7877fe640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc7881558a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc780002230 tx=0x7fc78000cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780017070 con 0x7fc788072580 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc780003da0 con 0x7fc788072580 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7881ec720 con 0x7fc788072580 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800052b0 con 0x7fc788072580 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.137+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc788154290 con 0x7fc788072580 2024-04-25T01:27:35.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.139+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc788075550 con 0x7fc788072580 2024-04-25T01:27:35.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.139+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc78001a070 con 0x7fc788072580 2024-04-25T01:27:35.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.139+0000 7fc76ffff640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc77406be30 0x7fc77406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.140+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc78008a800 con 0x7fc788072580 2024-04-25T01:27:35.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.140+0000 7fc786ffd640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc77406be30 0x7fc77406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.140+0000 7fc786ffd640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc77406be30 0x7fc77406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc778005880 tx=0x7fc778009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:35.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.144+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc788155de0 con 0x7fc788072580 2024-04-25T01:27:35.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.320+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7881eca00 con 0x7fc77406be30 2024-04-25T01:27:35.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.328+0000 7fc76ffff640 1 -- 172.21.15.129:0/3132232705 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333873 (secure 0 0 0) 0x7fc7881eca00 con 0x7fc77406be30 2024-04-25T01:27:35.310 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:35.312 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:27:35.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc77406be30 msgr2=0x7fc77406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc77406be30 0x7fc77406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc778005880 tx=0x7fc778009040 comp rx=0 tx=0).stop 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 msgr2=0x7fc7881558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc7881558a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc780002230 tx=0x7fc78000cbe0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 shutdown_connections 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc77406be30 0x7fc77406e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc788072580 0x7fc7881558a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc78806e4c0 0x7fc788155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 --2- 172.21.15.129:0/3132232705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc78806c2f0 0x7fc7881ebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 >> 172.21.15.129:0/3132232705 conn(0x7fc7880c3c80 msgr2=0x7fc7880c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:35.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 shutdown_connections 2024-04-25T01:27:35.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:35.331+0000 7fc78da15640 1 -- 172.21.15.129:0/3132232705 wait complete. 2024-04-25T01:27:35.321 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":808,"stamp":"2024-04-25T01:27:35.059055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231085285,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":15248,"num_read_kb":3072299,"num_write":423212,"num_write_kb":32362834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":588947,"ondisk_log_size":588947,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7983664,"kb_used_data":453680,"kb_used_omap":12,"kb_used_meta":7529907,"kb_avail":746991056,"statfs":{"total":773094113280,"available":764918841344,"internally_reserved":0,"allocated":464568320,"data_stored":463122826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7710625328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-221451310,"num_objects":-1338,"num_object_clones":0,"num_object_copies":-4024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1338,"num_whiteouts":0,"num_read":11,"num_read_kb":883,"num_write":2167,"num_write_kb":29642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-113,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002843"},"pg_stats":[{"pgid":"2.1f","version":"27'2276","reported_seq":2314,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.371340+0000","last_change":"2024-04-25T01:01:32.399292+0000","last_active":"2024-04-25T01:27:26.371340+0000","last_peered":"2024-04-25T01:27:26.371340+0000","last_clean":"2024-04-25T01:27:26.371340+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:27:26.371340+0000","last_undegraded":"2024-04-25T01:27:26.371340+0000","last_fullsized":"2024-04-25T01:27:26.371340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2276,"log_dups_size":0,"ondisk_log_size":2276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2285,"num_write_kb":17945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2549","reported_seq":2586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.059193+0000","last_change":"2024-04-25T01:01:33.400703+0000","last_active":"2024-04-25T01:27:29.059193+0000","last_peered":"2024-04-25T01:27:29.059193+0000","last_clean":"2024-04-25T01:27:29.059193+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:27:29.059193+0000","last_undegraded":"2024-04-25T01:27:29.059193+0000","last_fullsized":"2024-04-25T01:27:29.059193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9461","reported_seq":9780,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.546839+0000","last_change":"2024-04-25T01:01:37.442052+0000","last_active":"2024-04-25T01:27:29.546839+0000","last_peered":"2024-04-25T01:27:29.546839+0000","last_clean":"2024-04-25T01:27:29.546839+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:27:29.546839+0000","last_undegraded":"2024-04-25T01:27:29.546839+0000","last_fullsized":"2024-04-25T01:27:29.546839+0000","mapping_epoch":23,"log_start":"27'3202","ondisk_log_start":"27'3202","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":290,"num_read_kb":73660,"num_write":6825,"num_write_kb":1007677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6106","reported_seq":6145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.400413+0000","last_change":"2024-04-25T01:01:32.395435+0000","last_active":"2024-04-25T01:27:26.400413+0000","last_peered":"2024-04-25T01:27:26.400413+0000","last_clean":"2024-04-25T01:27:26.400413+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:27:26.400413+0000","last_undegraded":"2024-04-25T01:27:26.400413+0000","last_fullsized":"2024-04-25T01:27:26.400413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6106,"log_dups_size":0,"ondisk_log_size":6106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6114,"num_write_kb":49190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.426100+0000","last_change":"2024-04-25T01:01:33.409680+0000","last_active":"2024-04-25T01:27:29.426100+0000","last_peered":"2024-04-25T01:27:29.426100+0000","last_clean":"2024-04-25T01:27:29.426100+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:27:29.426100+0000","last_undegraded":"2024-04-25T01:27:29.426100+0000","last_fullsized":"2024-04-25T01:27:29.426100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:30.981934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9580","reported_seq":10037,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.434400+0000","last_change":"2024-04-25T01:01:37.435409+0000","last_active":"2024-04-25T01:27:29.434400+0000","last_peered":"2024-04-25T01:27:29.434400+0000","last_clean":"2024-04-25T01:27:29.434400+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:27:29.434400+0000","last_undegraded":"2024-04-25T01:27:29.434400+0000","last_fullsized":"2024-04-25T01:27:29.434400+0000","mapping_epoch":23,"log_start":"27'2431","ondisk_log_start":"27'2431","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7149,"log_dups_size":0,"ondisk_log_size":7149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:51.232882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":429,"num_read_kb":108966,"num_write":6832,"num_write_kb":1087543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9179","reported_seq":9308,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.351815+0000","last_change":"2024-04-25T01:01:32.398317+0000","last_active":"2024-04-25T01:27:28.351815+0000","last_peered":"2024-04-25T01:27:28.351815+0000","last_clean":"2024-04-25T01:27:28.351815+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:27:28.351815+0000","last_undegraded":"2024-04-25T01:27:28.351815+0000","last_fullsized":"2024-04-25T01:27:28.351815+0000","mapping_epoch":18,"log_start":"27'2200","ondisk_log_start":"27'2200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6979,"log_dups_size":0,"ondisk_log_size":6979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:09.032028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4982608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":93,"num_read_kb":825,"num_write":9172,"num_write_kb":76484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2632","reported_seq":2666,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.923693+0000","last_change":"2024-04-25T01:01:33.411236+0000","last_active":"2024-04-25T01:27:28.923693+0000","last_peered":"2024-04-25T01:27:28.923693+0000","last_clean":"2024-04-25T01:27:28.923693+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:27:28.923693+0000","last_undegraded":"2024-04-25T01:27:28.923693+0000","last_fullsized":"2024-04-25T01:27:28.923693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2632,"log_dups_size":0,"ondisk_log_size":2632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9168","reported_seq":9785,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.644264+0000","last_change":"2024-04-25T01:01:37.437217+0000","last_active":"2024-04-25T01:27:29.644264+0000","last_peered":"2024-04-25T01:27:29.644264+0000","last_clean":"2024-04-25T01:27:29.644264+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:27:29.644264+0000","last_undegraded":"2024-04-25T01:27:29.644264+0000","last_fullsized":"2024-04-25T01:27:29.644264+0000","mapping_epoch":23,"log_start":"27'2014","ondisk_log_start":"27'2014","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:14.849230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6458,"num_write_kb":913780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4331","reported_seq":4367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.374958+0000","last_change":"2024-04-25T01:01:32.400630+0000","last_active":"2024-04-25T01:27:26.374958+0000","last_peered":"2024-04-25T01:27:26.374958+0000","last_clean":"2024-04-25T01:27:26.374958+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:27:26.374958+0000","last_undegraded":"2024-04-25T01:27:26.374958+0000","last_fullsized":"2024-04-25T01:27:26.374958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4333,"num_write_kb":43749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2569","reported_seq":2603,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.515668+0000","last_change":"2024-04-25T01:01:33.410437+0000","last_active":"2024-04-25T01:27:29.515668+0000","last_peered":"2024-04-25T01:27:29.515668+0000","last_clean":"2024-04-25T01:27:29.515668+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:27:29.515668+0000","last_undegraded":"2024-04-25T01:27:29.515668+0000","last_fullsized":"2024-04-25T01:27:29.515668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.818578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8842","reported_seq":9253,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.842697+0000","last_change":"2024-04-25T01:01:37.431825+0000","last_active":"2024-04-25T01:27:28.842697+0000","last_peered":"2024-04-25T01:27:28.842697+0000","last_clean":"2024-04-25T01:27:28.842697+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:27:28.842697+0000","last_undegraded":"2024-04-25T01:27:28.842697+0000","last_fullsized":"2024-04-25T01:27:28.842697+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7232,"log_dups_size":0,"ondisk_log_size":7232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:27.230081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":383,"num_read_kb":97282,"num_write":6140,"num_write_kb":837273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2555","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.596401+0000","last_change":"2024-04-25T01:01:33.400926+0000","last_active":"2024-04-25T01:27:28.596401+0000","last_peered":"2024-04-25T01:27:28.596401+0000","last_clean":"2024-04-25T01:27:28.596401+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:27:28.596401+0000","last_undegraded":"2024-04-25T01:27:28.596401+0000","last_fullsized":"2024-04-25T01:27:28.596401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6999","reported_seq":7036,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.366224+0000","last_change":"2024-04-25T01:01:32.401003+0000","last_active":"2024-04-25T01:27:26.366224+0000","last_peered":"2024-04-25T01:27:26.366224+0000","last_clean":"2024-04-25T01:27:26.366224+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:27:26.366224+0000","last_undegraded":"2024-04-25T01:27:26.366224+0000","last_fullsized":"2024-04-25T01:27:26.366224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6999,"log_dups_size":0,"ondisk_log_size":6999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:17.815635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6999,"num_write_kb":53860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9960","reported_seq":10582,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.809241+0000","last_change":"2024-04-25T01:01:37.437245+0000","last_active":"2024-04-25T01:27:28.809241+0000","last_peered":"2024-04-25T01:27:28.809241+0000","last_clean":"2024-04-25T01:27:28.809241+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:27:28.809241+0000","last_undegraded":"2024-04-25T01:27:28.809241+0000","last_fullsized":"2024-04-25T01:27:28.809241+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7230,"log_dups_size":0,"ondisk_log_size":7230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:27.528783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":593,"num_read_kb":150622,"num_write":7206,"num_write_kb":1017663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2576","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.273465+0000","last_change":"2024-04-25T01:01:33.401467+0000","last_active":"2024-04-25T01:27:29.273465+0000","last_peered":"2024-04-25T01:27:29.273465+0000","last_clean":"2024-04-25T01:27:29.273465+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:27:29.273465+0000","last_undegraded":"2024-04-25T01:27:29.273465+0000","last_fullsized":"2024-04-25T01:27:29.273465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7175","reported_seq":7213,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.390959+0000","last_change":"2024-04-25T01:01:32.396827+0000","last_active":"2024-04-25T01:27:26.390959+0000","last_peered":"2024-04-25T01:27:26.390959+0000","last_clean":"2024-04-25T01:27:26.390959+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:27:26.390959+0000","last_undegraded":"2024-04-25T01:27:26.390959+0000","last_fullsized":"2024-04-25T01:27:26.390959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7178,"num_write_kb":58045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9188","reported_seq":9684,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.178147+0000","last_change":"2024-04-25T01:01:37.440186+0000","last_active":"2024-04-25T01:27:29.178147+0000","last_peered":"2024-04-25T01:27:29.178147+0000","last_clean":"2024-04-25T01:27:29.178147+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:27:29.178147+0000","last_undegraded":"2024-04-25T01:27:29.178147+0000","last_fullsized":"2024-04-25T01:27:29.178147+0000","mapping_epoch":23,"log_start":"27'2033","ondisk_log_start":"27'2033","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7155,"log_dups_size":0,"ondisk_log_size":7155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:35.146217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":118618,"num_write":6486,"num_write_kb":966622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2640","reported_seq":2676,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.872940+0000","last_change":"2024-04-25T01:01:33.397715+0000","last_active":"2024-04-25T01:27:28.872940+0000","last_peered":"2024-04-25T01:27:28.872940+0000","last_clean":"2024-04-25T01:27:28.872940+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:27:28.872940+0000","last_undegraded":"2024-04-25T01:27:28.872940+0000","last_fullsized":"2024-04-25T01:27:28.872940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3548","reported_seq":3586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.359257+0000","last_change":"2024-04-25T01:01:32.402614+0000","last_active":"2024-04-25T01:27:26.359257+0000","last_peered":"2024-04-25T01:27:26.359257+0000","last_clean":"2024-04-25T01:27:26.359257+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:27:26.359257+0000","last_undegraded":"2024-04-25T01:27:26.359257+0000","last_fullsized":"2024-04-25T01:27:26.359257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":26965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'8991","reported_seq":9507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.151524+0000","last_change":"2024-04-25T01:02:13.748397+0000","last_active":"2024-04-25T01:27:29.151524+0000","last_peered":"2024-04-25T01:27:29.151524+0000","last_clean":"2024-04-25T01:27:29.151524+0000","last_became_active":"2024-04-25T01:02:13.748037+0000","last_became_peered":"2024-04-25T01:02:13.748037+0000","last_unstale":"2024-04-25T01:27:29.151524+0000","last_undegraded":"2024-04-25T01:27:29.151524+0000","last_fullsized":"2024-04-25T01:27:29.151524+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":8991,"log_dups_size":0,"ondisk_log_size":8991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6426,"num_write_kb":893296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2544","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.017997+0000","last_change":"2024-04-25T01:01:33.401665+0000","last_active":"2024-04-25T01:27:29.017997+0000","last_peered":"2024-04-25T01:27:29.017997+0000","last_clean":"2024-04-25T01:27:29.017997+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:27:29.017997+0000","last_undegraded":"2024-04-25T01:27:29.017997+0000","last_fullsized":"2024-04-25T01:27:29.017997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2989","reported_seq":3026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.361419+0000","last_change":"2024-04-25T01:01:32.399234+0000","last_active":"2024-04-25T01:27:26.361419+0000","last_peered":"2024-04-25T01:27:26.361419+0000","last_clean":"2024-04-25T01:27:26.361419+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:27:26.361419+0000","last_undegraded":"2024-04-25T01:27:26.361419+0000","last_fullsized":"2024-04-25T01:27:26.361419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2989,"log_dups_size":0,"ondisk_log_size":2989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2993,"num_write_kb":22443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9534","reported_seq":9756,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.987441+0000","last_change":"2024-04-25T01:01:37.436125+0000","last_active":"2024-04-25T01:27:28.987441+0000","last_peered":"2024-04-25T01:27:28.987441+0000","last_clean":"2024-04-25T01:27:28.987441+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:27:28.987441+0000","last_undegraded":"2024-04-25T01:27:28.987441+0000","last_fullsized":"2024-04-25T01:27:28.987441+0000","mapping_epoch":23,"log_start":"27'2346","ondisk_log_start":"27'2346","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7188,"log_dups_size":0,"ondisk_log_size":7188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:39.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":49022,"num_write":6898,"num_write_kb":954026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2508","reported_seq":2543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.080176+0000","last_change":"2024-04-25T01:01:33.399179+0000","last_active":"2024-04-25T01:27:29.080176+0000","last_peered":"2024-04-25T01:27:29.080176+0000","last_clean":"2024-04-25T01:27:29.080176+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:27:29.080176+0000","last_undegraded":"2024-04-25T01:27:29.080176+0000","last_fullsized":"2024-04-25T01:27:29.080176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:39.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4624","reported_seq":4663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.565041+0000","last_change":"2024-04-25T01:01:32.402668+0000","last_active":"2024-04-25T01:27:34.565041+0000","last_peered":"2024-04-25T01:27:34.565041+0000","last_clean":"2024-04-25T01:27:34.565041+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:27:34.565041+0000","last_undegraded":"2024-04-25T01:27:34.565041+0000","last_fullsized":"2024-04-25T01:27:34.565041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4624,"log_dups_size":0,"ondisk_log_size":4624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4622,"num_write_kb":39981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9265","reported_seq":9332,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.194304+0000","last_change":"2024-04-25T01:01:37.430026+0000","last_active":"2024-04-25T01:27:29.194304+0000","last_peered":"2024-04-25T01:27:29.194304+0000","last_clean":"2024-04-25T01:27:29.194304+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:27:29.194304+0000","last_undegraded":"2024-04-25T01:27:29.194304+0000","last_fullsized":"2024-04-25T01:27:29.194304+0000","mapping_epoch":23,"log_start":"27'2105","ondisk_log_start":"27'2105","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7160,"log_dups_size":0,"ondisk_log_size":7160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:10.439027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":9398,"num_write":6620,"num_write_kb":863134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2500","reported_seq":2536,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.415712+0000","last_change":"2024-04-25T01:01:33.403947+0000","last_active":"2024-04-25T01:27:29.415712+0000","last_peered":"2024-04-25T01:27:29.415712+0000","last_clean":"2024-04-25T01:27:29.415712+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:27:29.415712+0000","last_undegraded":"2024-04-25T01:27:29.415712+0000","last_fullsized":"2024-04-25T01:27:29.415712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6903","reported_seq":7605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:32.292941+0000","last_change":"2024-04-25T01:01:32.394603+0000","last_active":"2024-04-25T01:27:32.292941+0000","last_peered":"2024-04-25T01:27:32.292941+0000","last_clean":"2024-04-25T01:27:32.292941+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:27:32.292941+0000","last_undegraded":"2024-04-25T01:27:32.292941+0000","last_fullsized":"2024-04-25T01:27:32.292941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6903,"log_dups_size":0,"ondisk_log_size":6903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":663,"num_read_kb":4517,"num_write":9705,"num_write_kb":61055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9005","reported_seq":9078,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.527570+0000","last_change":"2024-04-25T01:01:37.434568+0000","last_active":"2024-04-25T01:27:29.527570+0000","last_peered":"2024-04-25T01:27:29.527570+0000","last_clean":"2024-04-25T01:27:29.527570+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:27:29.527570+0000","last_undegraded":"2024-04-25T01:27:29.527570+0000","last_fullsized":"2024-04-25T01:27:29.527570+0000","mapping_epoch":23,"log_start":"27'2722","ondisk_log_start":"27'2722","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":10668,"num_write":6336,"num_write_kb":893375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2596","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.629300+0000","last_change":"2024-04-25T01:01:33.399282+0000","last_active":"2024-04-25T01:27:29.629300+0000","last_peered":"2024-04-25T01:27:29.629300+0000","last_clean":"2024-04-25T01:27:29.629300+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:27:29.629300+0000","last_undegraded":"2024-04-25T01:27:29.629300+0000","last_fullsized":"2024-04-25T01:27:29.629300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6064","reported_seq":6102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322120+0000","last_change":"2024-04-25T01:01:32.400350+0000","last_active":"2024-04-25T01:27:26.322120+0000","last_peered":"2024-04-25T01:27:26.322120+0000","last_clean":"2024-04-25T01:27:26.322120+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:27:26.322120+0000","last_undegraded":"2024-04-25T01:27:26.322120+0000","last_fullsized":"2024-04-25T01:27:26.322120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6064,"log_dups_size":0,"ondisk_log_size":6064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:25.349102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6062,"num_write_kb":49091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9822","reported_seq":10334,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.474257+0000","last_change":"2024-04-25T01:01:37.442017+0000","last_active":"2024-04-25T01:27:29.474257+0000","last_peered":"2024-04-25T01:27:29.474257+0000","last_clean":"2024-04-25T01:27:29.474257+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:27:29.474257+0000","last_undegraded":"2024-04-25T01:27:29.474257+0000","last_fullsized":"2024-04-25T01:27:29.474257+0000","mapping_epoch":23,"log_start":"27'2453","ondisk_log_start":"27'2453","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7369,"log_dups_size":0,"ondisk_log_size":7369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:39.164226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":482,"num_read_kb":122428,"num_write":7085,"num_write_kb":850051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2501","reported_seq":2537,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.609108+0000","last_change":"2024-04-25T01:01:33.406633+0000","last_active":"2024-04-25T01:27:28.609108+0000","last_peered":"2024-04-25T01:27:28.609108+0000","last_clean":"2024-04-25T01:27:28.609108+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:27:28.609108+0000","last_undegraded":"2024-04-25T01:27:28.609108+0000","last_fullsized":"2024-04-25T01:27:28.609108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2501,"log_dups_size":0,"ondisk_log_size":2501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7885","reported_seq":7924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.391761+0000","last_change":"2024-04-25T01:01:32.394115+0000","last_active":"2024-04-25T01:27:26.391761+0000","last_peered":"2024-04-25T01:27:26.391761+0000","last_clean":"2024-04-25T01:27:26.391761+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:27:26.391761+0000","last_undegraded":"2024-04-25T01:27:26.391761+0000","last_fullsized":"2024-04-25T01:27:26.391761+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7873,"num_write_kb":70555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9288","reported_seq":9779,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.410700+0000","last_change":"2024-04-25T01:01:37.435146+0000","last_active":"2024-04-25T01:27:29.410700+0000","last_peered":"2024-04-25T01:27:29.410700+0000","last_clean":"2024-04-25T01:27:29.410700+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:27:29.410700+0000","last_undegraded":"2024-04-25T01:27:29.410700+0000","last_fullsized":"2024-04-25T01:27:29.410700+0000","mapping_epoch":23,"log_start":"27'1900","ondisk_log_start":"27'1900","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7388,"log_dups_size":0,"ondisk_log_size":7388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":460,"num_read_kb":116840,"num_write":6608,"num_write_kb":921401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2540","reported_seq":2576,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.447522+0000","last_change":"2024-04-25T01:01:33.400607+0000","last_active":"2024-04-25T01:27:29.447522+0000","last_peered":"2024-04-25T01:27:29.447522+0000","last_clean":"2024-04-25T01:27:29.447522+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:27:29.447522+0000","last_undegraded":"2024-04-25T01:27:29.447522+0000","last_fullsized":"2024-04-25T01:27:29.447522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2540,"log_dups_size":0,"ondisk_log_size":2540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:18.821246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8861","reported_seq":8899,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.385459+0000","last_change":"2024-04-25T01:01:32.397491+0000","last_active":"2024-04-25T01:27:28.385459+0000","last_peered":"2024-04-25T01:27:28.385459+0000","last_clean":"2024-04-25T01:27:28.385459+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:27:28.385459+0000","last_undegraded":"2024-04-25T01:27:28.385459+0000","last_fullsized":"2024-04-25T01:27:28.385459+0000","mapping_epoch":18,"log_start":"27'1500","ondisk_log_start":"27'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7361,"log_dups_size":0,"ondisk_log_size":7361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:43.328498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7407031,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8858,"num_write_kb":62470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9932","reported_seq":10087,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.631317+0000","last_change":"2024-04-25T01:01:37.433197+0000","last_active":"2024-04-25T01:27:29.631317+0000","last_peered":"2024-04-25T01:27:29.631317+0000","last_clean":"2024-04-25T01:27:29.631317+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:27:29.631317+0000","last_undegraded":"2024-04-25T01:27:29.631317+0000","last_fullsized":"2024-04-25T01:27:29.631317+0000","mapping_epoch":23,"log_start":"27'2523","ondisk_log_start":"27'2523","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":31496,"num_write":7238,"num_write_kb":913890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2545","reported_seq":2582,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.264281+0000","last_change":"2024-04-25T01:01:33.394379+0000","last_active":"2024-04-25T01:27:29.264281+0000","last_peered":"2024-04-25T01:27:29.264281+0000","last_clean":"2024-04-25T01:27:29.264281+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:27:29.264281+0000","last_undegraded":"2024-04-25T01:27:29.264281+0000","last_fullsized":"2024-04-25T01:27:29.264281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2545,"log_dups_size":0,"ondisk_log_size":2545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7134","reported_seq":8084,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:21.230065+0000","last_change":"2024-04-25T01:01:32.392599+0000","last_active":"2024-04-25T01:27:21.230065+0000","last_peered":"2024-04-25T01:27:21.230065+0000","last_clean":"2024-04-25T01:27:21.230065+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:27:21.230065+0000","last_undegraded":"2024-04-25T01:27:21.230065+0000","last_fullsized":"2024-04-25T01:27:21.230065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":912,"num_read_kb":4677,"num_write":7122,"num_write_kb":61966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9066","reported_seq":9492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.501251+0000","last_change":"2024-04-25T01:01:37.436680+0000","last_active":"2024-04-25T01:27:29.501251+0000","last_peered":"2024-04-25T01:27:29.501251+0000","last_clean":"2024-04-25T01:27:29.501251+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:27:29.501251+0000","last_undegraded":"2024-04-25T01:27:29.501251+0000","last_fullsized":"2024-04-25T01:27:29.501251+0000","mapping_epoch":23,"log_start":"27'1700","ondisk_log_start":"27'1700","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":100584,"num_write":6328,"num_write_kb":903694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2479","reported_seq":2515,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.993214+0000","last_change":"2024-04-25T01:01:33.407781+0000","last_active":"2024-04-25T01:27:28.993214+0000","last_peered":"2024-04-25T01:27:28.993214+0000","last_clean":"2024-04-25T01:27:28.993214+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:27:28.993214+0000","last_undegraded":"2024-04-25T01:27:28.993214+0000","last_fullsized":"2024-04-25T01:27:28.993214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:33.079494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'6806","reported_seq":6844,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.391028+0000","last_change":"2024-04-25T01:01:32.400628+0000","last_active":"2024-04-25T01:27:26.391028+0000","last_peered":"2024-04-25T01:27:26.391028+0000","last_clean":"2024-04-25T01:27:26.391028+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:27:26.391028+0000","last_undegraded":"2024-04-25T01:27:26.391028+0000","last_fullsized":"2024-04-25T01:27:26.391028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6806,"num_write_kb":57773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9197","reported_seq":9555,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.080380+0000","last_change":"2024-04-25T01:01:37.440856+0000","last_active":"2024-04-25T01:27:29.080380+0000","last_peered":"2024-04-25T01:27:29.080380+0000","last_clean":"2024-04-25T01:27:29.080380+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:27:29.080380+0000","last_undegraded":"2024-04-25T01:27:29.080380+0000","last_fullsized":"2024-04-25T01:27:29.080380+0000","mapping_epoch":23,"log_start":"27'1826","ondisk_log_start":"27'1826","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:41.474999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":83566,"num_write":6535,"num_write_kb":1105233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2598","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.342613+0000","last_change":"2024-04-25T01:01:33.414371+0000","last_active":"2024-04-25T01:27:29.342613+0000","last_peered":"2024-04-25T01:27:29.342613+0000","last_clean":"2024-04-25T01:27:29.342613+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:27:29.342613+0000","last_undegraded":"2024-04-25T01:27:29.342613+0000","last_fullsized":"2024-04-25T01:27:29.342613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4347","reported_seq":4385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.334344+0000","last_change":"2024-04-25T01:01:32.397634+0000","last_active":"2024-04-25T01:27:26.334344+0000","last_peered":"2024-04-25T01:27:26.334344+0000","last_clean":"2024-04-25T01:27:26.334344+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:27:26.334344+0000","last_undegraded":"2024-04-25T01:27:26.334344+0000","last_fullsized":"2024-04-25T01:27:26.334344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:55.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4347,"num_write_kb":39756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:25:30.989614+0000","last_change":"2024-04-25T01:01:22.652898+0000","last_active":"2024-04-25T01:25:30.989614+0000","last_peered":"2024-04-25T01:25:30.989614+0000","last_clean":"2024-04-25T01:25:30.989614+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:25:30.989614+0000","last_undegraded":"2024-04-25T01:25:30.989614+0000","last_fullsized":"2024-04-25T01:25:30.989614+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:21.321421+0000","last_clean_scrub_stamp":"2024-04-25T01:01:21.321421+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:05.234757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'9541","reported_seq":9803,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.861422+0000","last_change":"2024-04-25T01:01:37.431008+0000","last_active":"2024-04-25T01:27:28.861422+0000","last_peered":"2024-04-25T01:27:28.861422+0000","last_clean":"2024-04-25T01:27:28.861422+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:27:28.861422+0000","last_undegraded":"2024-04-25T01:27:28.861422+0000","last_fullsized":"2024-04-25T01:27:28.861422+0000","mapping_epoch":23,"log_start":"27'1566","ondisk_log_start":"27'1566","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":58674,"num_write":6904,"num_write_kb":915015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2524","reported_seq":2559,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.703146+0000","last_change":"2024-04-25T01:01:33.409962+0000","last_active":"2024-04-25T01:27:28.703146+0000","last_peered":"2024-04-25T01:27:28.703146+0000","last_clean":"2024-04-25T01:27:28.703146+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:27:28.703146+0000","last_undegraded":"2024-04-25T01:27:28.703146+0000","last_fullsized":"2024-04-25T01:27:28.703146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7384","reported_seq":7421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.375187+0000","last_change":"2024-04-25T01:01:32.399586+0000","last_active":"2024-04-25T01:27:26.375187+0000","last_peered":"2024-04-25T01:27:26.375187+0000","last_clean":"2024-04-25T01:27:26.375187+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:27:26.375187+0000","last_undegraded":"2024-04-25T01:27:26.375187+0000","last_fullsized":"2024-04-25T01:27:26.375187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:29.057967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7389,"num_write_kb":58188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9283","reported_seq":9788,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.642278+0000","last_change":"2024-04-25T01:02:13.830001+0000","last_active":"2024-04-25T01:27:28.642278+0000","last_peered":"2024-04-25T01:27:28.642278+0000","last_clean":"2024-04-25T01:27:28.642278+0000","last_became_active":"2024-04-25T01:02:13.829268+0000","last_became_peered":"2024-04-25T01:02:13.829268+0000","last_unstale":"2024-04-25T01:27:28.642278+0000","last_undegraded":"2024-04-25T01:27:28.642278+0000","last_fullsized":"2024-04-25T01:27:28.642278+0000","mapping_epoch":26,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":468,"num_read_kb":118872,"num_write":6604,"num_write_kb":925635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2554","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.761595+0000","last_change":"2024-04-25T01:01:33.411811+0000","last_active":"2024-04-25T01:27:28.761595+0000","last_peered":"2024-04-25T01:27:28.761595+0000","last_clean":"2024-04-25T01:27:28.761595+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:27:28.761595+0000","last_undegraded":"2024-04-25T01:27:28.761595+0000","last_fullsized":"2024-04-25T01:27:28.761595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7034","reported_seq":7072,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.397227+0000","last_change":"2024-04-25T01:01:32.391577+0000","last_active":"2024-04-25T01:27:26.397227+0000","last_peered":"2024-04-25T01:27:26.397227+0000","last_clean":"2024-04-25T01:27:26.397227+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:27:26.397227+0000","last_undegraded":"2024-04-25T01:27:26.397227+0000","last_fullsized":"2024-04-25T01:27:26.397227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7035,"num_write_kb":58075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9471","reported_seq":10164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.259281+0000","last_change":"2024-04-25T01:01:37.440958+0000","last_active":"2024-04-25T01:27:29.259281+0000","last_peered":"2024-04-25T01:27:29.259281+0000","last_clean":"2024-04-25T01:27:29.259281+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:27:29.259281+0000","last_undegraded":"2024-04-25T01:27:29.259281+0000","last_fullsized":"2024-04-25T01:27:29.259281+0000","mapping_epoch":23,"log_start":"27'2138","ondisk_log_start":"27'2138","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7333,"log_dups_size":0,"ondisk_log_size":7333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:39.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":664,"num_read_kb":168656,"num_write":6719,"num_write_kb":895554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2424","reported_seq":2459,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.094758+0000","last_change":"2024-04-25T01:01:33.390013+0000","last_active":"2024-04-25T01:27:29.094758+0000","last_peered":"2024-04-25T01:27:29.094758+0000","last_clean":"2024-04-25T01:27:29.094758+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:27:29.094758+0000","last_undegraded":"2024-04-25T01:27:29.094758+0000","last_fullsized":"2024-04-25T01:27:29.094758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2424,"log_dups_size":0,"ondisk_log_size":2424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:35.875581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8741","reported_seq":8778,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.401067+0000","last_change":"2024-04-25T01:01:32.393161+0000","last_active":"2024-04-25T01:27:26.401067+0000","last_peered":"2024-04-25T01:27:26.401067+0000","last_clean":"2024-04-25T01:27:26.401067+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:27:26.401067+0000","last_undegraded":"2024-04-25T01:27:26.401067+0000","last_fullsized":"2024-04-25T01:27:26.401067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":8741,"log_dups_size":0,"ondisk_log_size":8741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:39.255790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8749,"num_write_kb":79319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9302","reported_seq":10366,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.855328+0000","last_change":"2024-04-25T01:01:37.431520+0000","last_active":"2024-04-25T01:27:28.855328+0000","last_peered":"2024-04-25T01:27:28.855328+0000","last_clean":"2024-04-25T01:27:28.855328+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:27:28.855328+0000","last_undegraded":"2024-04-25T01:27:28.855328+0000","last_fullsized":"2024-04-25T01:27:28.855328+0000","mapping_epoch":23,"log_start":"27'1886","ondisk_log_start":"27'1886","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:20.651479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1035,"num_read_kb":262890,"num_write":6564,"num_write_kb":1048321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2520","reported_seq":2555,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.870531+0000","last_change":"2024-04-25T01:01:33.390852+0000","last_active":"2024-04-25T01:27:28.870531+0000","last_peered":"2024-04-25T01:27:28.870531+0000","last_clean":"2024-04-25T01:27:28.870531+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:27:28.870531+0000","last_undegraded":"2024-04-25T01:27:28.870531+0000","last_fullsized":"2024-04-25T01:27:28.870531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5708","reported_seq":5745,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.330594+0000","last_change":"2024-04-25T01:01:32.394461+0000","last_active":"2024-04-25T01:27:26.330594+0000","last_peered":"2024-04-25T01:27:26.330594+0000","last_clean":"2024-04-25T01:27:26.330594+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:27:26.330594+0000","last_undegraded":"2024-04-25T01:27:26.330594+0000","last_fullsized":"2024-04-25T01:27:26.330594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:35:47.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":44693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'8871","reported_seq":9294,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.051984+0000","last_change":"2024-04-25T01:01:37.440499+0000","last_active":"2024-04-25T01:27:29.051984+0000","last_peered":"2024-04-25T01:27:29.051984+0000","last_clean":"2024-04-25T01:27:29.051984+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:27:29.051984+0000","last_undegraded":"2024-04-25T01:27:29.051984+0000","last_fullsized":"2024-04-25T01:27:29.051984+0000","mapping_epoch":23,"log_start":"27'1713","ondisk_log_start":"27'1713","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:55.852384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":394,"num_read_kb":100076,"num_write":6161,"num_write_kb":854384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2599","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.408304+0000","last_change":"2024-04-25T01:01:33.388734+0000","last_active":"2024-04-25T01:27:29.408304+0000","last_peered":"2024-04-25T01:27:29.408304+0000","last_clean":"2024-04-25T01:27:29.408304+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:27:29.408304+0000","last_undegraded":"2024-04-25T01:27:29.408304+0000","last_fullsized":"2024-04-25T01:27:29.408304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.115565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'5132","reported_seq":5169,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.356882+0000","last_change":"2024-04-25T01:01:32.394119+0000","last_active":"2024-04-25T01:27:26.356882+0000","last_peered":"2024-04-25T01:27:26.356882+0000","last_clean":"2024-04-25T01:27:26.356882+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:27:26.356882+0000","last_undegraded":"2024-04-25T01:27:26.356882+0000","last_fullsized":"2024-04-25T01:27:26.356882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5132,"log_dups_size":0,"ondisk_log_size":5132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5139,"num_write_kb":40185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9389","reported_seq":9725,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.360990+0000","last_change":"2024-04-25T01:01:37.436412+0000","last_active":"2024-04-25T01:27:29.360990+0000","last_peered":"2024-04-25T01:27:29.360990+0000","last_clean":"2024-04-25T01:27:29.360990+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:27:29.360990+0000","last_undegraded":"2024-04-25T01:27:29.360990+0000","last_fullsized":"2024-04-25T01:27:29.360990+0000","mapping_epoch":23,"log_start":"27'2023","ondisk_log_start":"27'2023","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.518322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":78232,"num_write":6705,"num_write_kb":1182057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2468","reported_seq":2504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.498338+0000","last_change":"2024-04-25T01:01:33.402567+0000","last_active":"2024-04-25T01:27:29.498338+0000","last_peered":"2024-04-25T01:27:29.498338+0000","last_clean":"2024-04-25T01:27:29.498338+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:27:29.498338+0000","last_undegraded":"2024-04-25T01:27:29.498338+0000","last_fullsized":"2024-04-25T01:27:29.498338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2468,"log_dups_size":0,"ondisk_log_size":2468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4844","reported_seq":4882,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.378359+0000","last_change":"2024-04-25T01:01:32.398586+0000","last_active":"2024-04-25T01:27:26.378359+0000","last_peered":"2024-04-25T01:27:26.378359+0000","last_clean":"2024-04-25T01:27:26.378359+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:27:26.378359+0000","last_undegraded":"2024-04-25T01:27:26.378359+0000","last_fullsized":"2024-04-25T01:27:26.378359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4848,"num_write_kb":44079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9127","reported_seq":9629,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.637455+0000","last_change":"2024-04-25T01:01:37.442642+0000","last_active":"2024-04-25T01:27:29.637455+0000","last_peered":"2024-04-25T01:27:29.637455+0000","last_clean":"2024-04-25T01:27:29.637455+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:27:29.637455+0000","last_undegraded":"2024-04-25T01:27:29.637455+0000","last_fullsized":"2024-04-25T01:27:29.637455+0000","mapping_epoch":23,"log_start":"27'2828","ondisk_log_start":"27'2828","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6299,"log_dups_size":0,"ondisk_log_size":6299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":473,"num_read_kb":120142,"num_write":6396,"num_write_kb":909662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2571","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.623157+0000","last_change":"2024-04-25T01:02:13.744191+0000","last_active":"2024-04-25T01:27:29.623157+0000","last_peered":"2024-04-25T01:27:29.623157+0000","last_clean":"2024-04-25T01:27:29.623157+0000","last_became_active":"2024-04-25T01:02:13.743832+0000","last_became_peered":"2024-04-25T01:02:13.743832+0000","last_unstale":"2024-04-25T01:27:29.623157+0000","last_undegraded":"2024-04-25T01:27:29.623157+0000","last_fullsized":"2024-04-25T01:27:29.623157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2571,"log_dups_size":0,"ondisk_log_size":2571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"27'2887","reported_seq":2924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.345321+0000","last_change":"2024-04-25T01:01:32.398039+0000","last_active":"2024-04-25T01:27:26.345321+0000","last_peered":"2024-04-25T01:27:26.345321+0000","last_clean":"2024-04-25T01:27:26.345321+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:27:26.345321+0000","last_undegraded":"2024-04-25T01:27:26.345321+0000","last_fullsized":"2024-04-25T01:27:26.345321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":2887,"log_dups_size":0,"ondisk_log_size":2887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:22:59.963159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2884,"num_write_kb":30449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9238","reported_seq":9482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.510976+0000","last_change":"2024-04-25T01:02:13.830353+0000","last_active":"2024-04-25T01:27:29.510976+0000","last_peered":"2024-04-25T01:27:29.510976+0000","last_clean":"2024-04-25T01:27:29.510976+0000","last_became_active":"2024-04-25T01:02:13.829847+0000","last_became_peered":"2024-04-25T01:02:13.829847+0000","last_unstale":"2024-04-25T01:27:29.510976+0000","last_undegraded":"2024-04-25T01:27:29.510976+0000","last_fullsized":"2024-04-25T01:27:29.510976+0000","mapping_epoch":26,"log_start":"27'2967","ondisk_log_start":"27'2967","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6271,"log_dups_size":0,"ondisk_log_size":6271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":207,"num_read_kb":52578,"num_write":6628,"num_write_kb":923125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2535","reported_seq":2570,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.194711+0000","last_change":"2024-04-25T01:01:33.404324+0000","last_active":"2024-04-25T01:27:29.194711+0000","last_peered":"2024-04-25T01:27:29.194711+0000","last_clean":"2024-04-25T01:27:29.194711+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:27:29.194711+0000","last_undegraded":"2024-04-25T01:27:29.194711+0000","last_fullsized":"2024-04-25T01:27:29.194711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2535,"log_dups_size":0,"ondisk_log_size":2535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:21.848018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6773","reported_seq":6810,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.367237+0000","last_change":"2024-04-25T01:01:32.391424+0000","last_active":"2024-04-25T01:27:26.367237+0000","last_peered":"2024-04-25T01:27:26.367237+0000","last_clean":"2024-04-25T01:27:26.367237+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:27:26.367237+0000","last_undegraded":"2024-04-25T01:27:26.367237+0000","last_fullsized":"2024-04-25T01:27:26.367237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6773,"log_dups_size":0,"ondisk_log_size":6773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:13.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6781,"num_write_kb":57696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9620","reported_seq":9997,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.803686+0000","last_change":"2024-04-25T01:01:37.438263+0000","last_active":"2024-04-25T01:27:28.803686+0000","last_peered":"2024-04-25T01:27:28.803686+0000","last_clean":"2024-04-25T01:27:28.803686+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:27:28.803686+0000","last_undegraded":"2024-04-25T01:27:28.803686+0000","last_fullsized":"2024-04-25T01:27:28.803686+0000","mapping_epoch":23,"log_start":"27'2214","ondisk_log_start":"27'2214","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7406,"log_dups_size":0,"ondisk_log_size":7406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:28.600372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":349,"num_read_kb":88646,"num_write":6988,"num_write_kb":1079633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2474","reported_seq":2508,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.762604+0000","last_change":"2024-04-25T01:01:33.396269+0000","last_active":"2024-04-25T01:27:28.762604+0000","last_peered":"2024-04-25T01:27:28.762604+0000","last_clean":"2024-04-25T01:27:28.762604+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:27:28.762604+0000","last_undegraded":"2024-04-25T01:27:28.762604+0000","last_fullsized":"2024-04-25T01:27:28.762604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2474,"log_dups_size":0,"ondisk_log_size":2474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:23.498131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10283","reported_seq":10914,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.596858+0000","last_change":"2024-04-25T01:01:32.394399+0000","last_active":"2024-04-25T01:27:29.596858+0000","last_peered":"2024-04-25T01:27:29.596858+0000","last_clean":"2024-04-25T01:27:29.596858+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:27:29.596858+0000","last_undegraded":"2024-04-25T01:27:29.596858+0000","last_fullsized":"2024-04-25T01:27:29.596858+0000","mapping_epoch":18,"log_start":"27'2300","ondisk_log_start":"27'2300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7983,"log_dups_size":0,"ondisk_log_size":7983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":594,"num_read_kb":4482,"num_write":10277,"num_write_kb":76426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9154","reported_seq":9496,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.324034+0000","last_change":"2024-04-25T01:01:37.431593+0000","last_active":"2024-04-25T01:27:29.324034+0000","last_peered":"2024-04-25T01:27:29.324034+0000","last_clean":"2024-04-25T01:27:29.324034+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:27:29.324034+0000","last_undegraded":"2024-04-25T01:27:29.324034+0000","last_fullsized":"2024-04-25T01:27:29.324034+0000","mapping_epoch":23,"log_start":"27'1834","ondisk_log_start":"27'1834","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7320,"log_dups_size":0,"ondisk_log_size":7320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":312,"num_read_kb":79248,"num_write":6491,"num_write_kb":938399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2554","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.136541+0000","last_change":"2024-04-25T01:01:33.388359+0000","last_active":"2024-04-25T01:27:29.136541+0000","last_peered":"2024-04-25T01:27:29.136541+0000","last_clean":"2024-04-25T01:27:29.136541+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:27:29.136541+0000","last_undegraded":"2024-04-25T01:27:29.136541+0000","last_fullsized":"2024-04-25T01:27:29.136541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6457","reported_seq":6494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.650698+0000","last_change":"2024-04-25T01:01:32.397719+0000","last_active":"2024-04-25T01:27:29.650698+0000","last_peered":"2024-04-25T01:27:29.650698+0000","last_clean":"2024-04-25T01:27:29.650698+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:27:29.650698+0000","last_undegraded":"2024-04-25T01:27:29.650698+0000","last_fullsized":"2024-04-25T01:27:29.650698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6457,"log_dups_size":0,"ondisk_log_size":6457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6472,"num_write_kb":41579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'8967","reported_seq":9504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.636183+0000","last_change":"2024-04-25T01:01:37.440890+0000","last_active":"2024-04-25T01:27:29.636183+0000","last_peered":"2024-04-25T01:27:29.636183+0000","last_clean":"2024-04-25T01:27:29.636183+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:27:29.636183+0000","last_undegraded":"2024-04-25T01:27:29.636183+0000","last_fullsized":"2024-04-25T01:27:29.636183+0000","mapping_epoch":23,"log_start":"27'1803","ondisk_log_start":"27'1803","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:25.079339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":508,"num_read_kb":129032,"num_write":6269,"num_write_kb":1000518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2578","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.594524+0000","last_change":"2024-04-25T01:01:33.403675+0000","last_active":"2024-04-25T01:27:29.594524+0000","last_peered":"2024-04-25T01:27:29.594524+0000","last_clean":"2024-04-25T01:27:29.594524+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:27:29.594524+0000","last_undegraded":"2024-04-25T01:27:29.594524+0000","last_fullsized":"2024-04-25T01:27:29.594524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:57.079371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6614","reported_seq":6650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.363784+0000","last_change":"2024-04-25T01:01:32.398495+0000","last_active":"2024-04-25T01:27:26.363784+0000","last_peered":"2024-04-25T01:27:26.363784+0000","last_clean":"2024-04-25T01:27:26.363784+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:27:26.363784+0000","last_undegraded":"2024-04-25T01:27:26.363784+0000","last_fullsized":"2024-04-25T01:27:26.363784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6614,"log_dups_size":0,"ondisk_log_size":6614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:20.804866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6608,"num_write_kb":53556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9229","reported_seq":9437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.588950+0000","last_change":"2024-04-25T01:01:37.433185+0000","last_active":"2024-04-25T01:27:29.588950+0000","last_peered":"2024-04-25T01:27:29.588950+0000","last_clean":"2024-04-25T01:27:29.588950+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:27:29.588950+0000","last_undegraded":"2024-04-25T01:27:29.588950+0000","last_fullsized":"2024-04-25T01:27:29.588950+0000","mapping_epoch":23,"log_start":"27'2910","ondisk_log_start":"27'2910","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6319,"log_dups_size":0,"ondisk_log_size":6319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6556,"num_write_kb":963248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2548","reported_seq":2582,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.304590+0000","last_change":"2024-04-25T01:01:33.413266+0000","last_active":"2024-04-25T01:27:29.304590+0000","last_peered":"2024-04-25T01:27:29.304590+0000","last_clean":"2024-04-25T01:27:29.304590+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:27:29.304590+0000","last_undegraded":"2024-04-25T01:27:29.304590+0000","last_fullsized":"2024-04-25T01:27:29.304590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:14:21.048686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6623","reported_seq":7479,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.368188+0000","last_change":"2024-04-25T01:01:32.398367+0000","last_active":"2024-04-25T01:27:26.368188+0000","last_peered":"2024-04-25T01:27:26.368188+0000","last_clean":"2024-04-25T01:27:26.368188+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:27:26.368188+0000","last_undegraded":"2024-04-25T01:27:26.368188+0000","last_fullsized":"2024-04-25T01:27:26.368188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6623,"log_dups_size":0,"ondisk_log_size":6623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":821,"num_read_kb":4618,"num_write":6625,"num_write_kb":49469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'10011","reported_seq":10076,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.570647+0000","last_change":"2024-04-25T01:01:37.441426+0000","last_active":"2024-04-25T01:27:29.570647+0000","last_peered":"2024-04-25T01:27:29.570647+0000","last_clean":"2024-04-25T01:27:29.570647+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:27:29.570647+0000","last_undegraded":"2024-04-25T01:27:29.570647+0000","last_fullsized":"2024-04-25T01:27:29.570647+0000","mapping_epoch":23,"log_start":"27'3717","ondisk_log_start":"27'3717","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6294,"log_dups_size":0,"ondisk_log_size":6294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":8890,"num_write":7291,"num_write_kb":1064218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9075","reported_seq":9332,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.082466+0000","last_change":"2024-04-25T01:01:37.444096+0000","last_active":"2024-04-25T01:27:29.082466+0000","last_peered":"2024-04-25T01:27:29.082466+0000","last_clean":"2024-04-25T01:27:29.082466+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:27:29.082466+0000","last_undegraded":"2024-04-25T01:27:29.082466+0000","last_fullsized":"2024-04-25T01:27:29.082466+0000","mapping_epoch":23,"log_start":"27'1719","ondisk_log_start":"27'1719","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:32.019183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":227,"num_read_kb":57658,"num_write":6476,"num_write_kb":909388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2493","reported_seq":2528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.538501+0000","last_change":"2024-04-25T01:01:33.393645+0000","last_active":"2024-04-25T01:27:29.538501+0000","last_peered":"2024-04-25T01:27:29.538501+0000","last_clean":"2024-04-25T01:27:29.538501+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:27:29.538501+0000","last_undegraded":"2024-04-25T01:27:29.538501+0000","last_fullsized":"2024-04-25T01:27:29.538501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9899","reported_seq":9935,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.322490+0000","last_change":"2024-04-25T01:01:32.399434+0000","last_active":"2024-04-25T01:27:26.322490+0000","last_peered":"2024-04-25T01:27:26.322490+0000","last_clean":"2024-04-25T01:27:26.322490+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:27:26.322490+0000","last_undegraded":"2024-04-25T01:27:26.322490+0000","last_fullsized":"2024-04-25T01:27:26.322490+0000","mapping_epoch":18,"log_start":"27'2500","ondisk_log_start":"27'2500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:12.113945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":68120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8969","reported_seq":9829,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.468636+0000","last_change":"2024-04-25T01:01:37.435974+0000","last_active":"2024-04-25T01:27:29.468636+0000","last_peered":"2024-04-25T01:27:29.468636+0000","last_clean":"2024-04-25T01:27:29.468636+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:27:29.468636+0000","last_undegraded":"2024-04-25T01:27:29.468636+0000","last_fullsized":"2024-04-25T01:27:29.468636+0000","mapping_epoch":23,"log_start":"27'1000","ondisk_log_start":"27'1000","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":829,"num_read_kb":210566,"num_write":6335,"num_write_kb":893076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2595","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.323194+0000","last_change":"2024-04-25T01:01:33.395029+0000","last_active":"2024-04-25T01:27:29.323194+0000","last_peered":"2024-04-25T01:27:29.323194+0000","last_clean":"2024-04-25T01:27:29.323194+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:27:29.323194+0000","last_undegraded":"2024-04-25T01:27:29.323194+0000","last_fullsized":"2024-04-25T01:27:29.323194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5344","reported_seq":5381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.377047+0000","last_change":"2024-04-25T01:01:32.391541+0000","last_active":"2024-04-25T01:27:26.377047+0000","last_peered":"2024-04-25T01:27:26.377047+0000","last_clean":"2024-04-25T01:27:26.377047+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:27:26.377047+0000","last_undegraded":"2024-04-25T01:27:26.377047+0000","last_fullsized":"2024-04-25T01:27:26.377047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5344,"num_write_kb":44512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9502","reported_seq":9539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.386731+0000","last_change":"2024-04-25T01:01:32.398646+0000","last_active":"2024-04-25T01:27:26.386731+0000","last_peered":"2024-04-25T01:27:26.386731+0000","last_clean":"2024-04-25T01:27:26.386731+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:27:26.386731+0000","last_undegraded":"2024-04-25T01:27:26.386731+0000","last_fullsized":"2024-04-25T01:27:26.386731+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":7902,"log_dups_size":0,"ondisk_log_size":7902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":9513,"num_write_kb":71968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2531","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.558531+0000","last_change":"2024-04-25T01:01:33.406389+0000","last_active":"2024-04-25T01:27:29.558531+0000","last_peered":"2024-04-25T01:27:29.558531+0000","last_clean":"2024-04-25T01:27:29.558531+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:27:29.558531+0000","last_undegraded":"2024-04-25T01:27:29.558531+0000","last_fullsized":"2024-04-25T01:27:29.558531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2531,"log_dups_size":0,"ondisk_log_size":2531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:23.870362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9470","reported_seq":9625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.045080+0000","last_change":"2024-04-25T01:01:37.438512+0000","last_active":"2024-04-25T01:27:29.045080+0000","last_peered":"2024-04-25T01:27:29.045080+0000","last_clean":"2024-04-25T01:27:29.045080+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:27:29.045080+0000","last_undegraded":"2024-04-25T01:27:29.045080+0000","last_fullsized":"2024-04-25T01:27:29.045080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:48.364761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":32258,"num_write":6759,"num_write_kb":1001286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8026","reported_seq":8063,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:26.384117+0000","last_change":"2024-04-25T01:01:32.400195+0000","last_active":"2024-04-25T01:27:26.384117+0000","last_peered":"2024-04-25T01:27:26.384117+0000","last_clean":"2024-04-25T01:27:26.384117+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:27:26.384117+0000","last_undegraded":"2024-04-25T01:27:26.384117+0000","last_fullsized":"2024-04-25T01:27:26.384117+0000","mapping_epoch":18,"log_start":"27'1700","ondisk_log_start":"27'1700","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:31.368719+0000","last_clean_scrub_stamp":"2024-04-25T01:01:31.368719+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8021,"num_write_kb":66584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2569","reported_seq":2603,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.787822+0000","last_change":"2024-04-25T01:01:33.387202+0000","last_active":"2024-04-25T01:27:28.787822+0000","last_peered":"2024-04-25T01:27:28.787822+0000","last_clean":"2024-04-25T01:27:28.787822+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:27:28.787822+0000","last_undegraded":"2024-04-25T01:27:28.787822+0000","last_fullsized":"2024-04-25T01:27:28.787822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:21.810476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9481","reported_seq":9878,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.600480+0000","last_change":"2024-04-25T01:01:37.440101+0000","last_active":"2024-04-25T01:27:29.600480+0000","last_peered":"2024-04-25T01:27:29.600480+0000","last_clean":"2024-04-25T01:27:29.600480+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:27:29.600480+0000","last_undegraded":"2024-04-25T01:27:29.600480+0000","last_fullsized":"2024-04-25T01:27:29.600480+0000","mapping_epoch":23,"log_start":"27'3225","ondisk_log_start":"27'3225","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:36.404982+0000","last_clean_scrub_stamp":"2024-04-25T01:01:36.404982+0000","objects_scrubbed":0,"log_size":6256,"log_dups_size":0,"ondisk_log_size":6256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":368,"num_read_kb":93472,"num_write":6767,"num_write_kb":1052015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2469","reported_seq":2506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.482237+0000","last_change":"2024-04-25T01:01:33.402799+0000","last_active":"2024-04-25T01:27:29.482237+0000","last_peered":"2024-04-25T01:27:29.482237+0000","last_clean":"2024-04-25T01:27:29.482237+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:27:29.482237+0000","last_undegraded":"2024-04-25T01:27:29.482237+0000","last_fullsized":"2024-04-25T01:27:29.482237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2598","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.111167+0000","last_change":"2024-04-25T01:01:33.400678+0000","last_active":"2024-04-25T01:27:29.111167+0000","last_peered":"2024-04-25T01:27:29.111167+0000","last_clean":"2024-04-25T01:27:29.111167+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:27:29.111167+0000","last_undegraded":"2024-04-25T01:27:29.111167+0000","last_fullsized":"2024-04-25T01:27:29.111167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:26.354712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2500","reported_seq":2536,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.338127+0000","last_change":"2024-04-25T01:01:33.411789+0000","last_active":"2024-04-25T01:27:29.338127+0000","last_peered":"2024-04-25T01:27:29.338127+0000","last_clean":"2024-04-25T01:27:29.338127+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:27:29.338127+0000","last_undegraded":"2024-04-25T01:27:29.338127+0000","last_fullsized":"2024-04-25T01:27:29.338127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2579","reported_seq":2615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.339759+0000","last_change":"2024-04-25T01:01:33.394777+0000","last_active":"2024-04-25T01:27:29.339759+0000","last_peered":"2024-04-25T01:27:29.339759+0000","last_clean":"2024-04-25T01:27:29.339759+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:27:29.339759+0000","last_undegraded":"2024-04-25T01:27:29.339759+0000","last_fullsized":"2024-04-25T01:27:29.339759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2579,"log_dups_size":0,"ondisk_log_size":2579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2580","reported_seq":2617,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.624696+0000","last_change":"2024-04-25T01:01:33.404514+0000","last_active":"2024-04-25T01:27:29.624696+0000","last_peered":"2024-04-25T01:27:29.624696+0000","last_clean":"2024-04-25T01:27:29.624696+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:27:29.624696+0000","last_undegraded":"2024-04-25T01:27:29.624696+0000","last_fullsized":"2024-04-25T01:27:29.624696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2456","reported_seq":2492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.857298+0000","last_change":"2024-04-25T01:01:33.392874+0000","last_active":"2024-04-25T01:27:28.857298+0000","last_peered":"2024-04-25T01:27:28.857298+0000","last_clean":"2024-04-25T01:27:28.857298+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:27:28.857298+0000","last_undegraded":"2024-04-25T01:27:28.857298+0000","last_fullsized":"2024-04-25T01:27:28.857298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2456,"log_dups_size":0,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2589","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.904089+0000","last_change":"2024-04-25T01:01:33.404926+0000","last_active":"2024-04-25T01:27:28.904089+0000","last_peered":"2024-04-25T01:27:28.904089+0000","last_clean":"2024-04-25T01:27:28.904089+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:27:28.904089+0000","last_undegraded":"2024-04-25T01:27:28.904089+0000","last_fullsized":"2024-04-25T01:27:28.904089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2556","reported_seq":2592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.407527+0000","last_change":"2024-04-25T01:01:33.405488+0000","last_active":"2024-04-25T01:27:29.407527+0000","last_peered":"2024-04-25T01:27:29.407527+0000","last_clean":"2024-04-25T01:27:29.407527+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:27:29.407527+0000","last_undegraded":"2024-04-25T01:27:29.407527+0000","last_fullsized":"2024-04-25T01:27:29.407527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2556,"log_dups_size":0,"ondisk_log_size":2556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2533","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.854644+0000","last_change":"2024-04-25T01:01:33.403641+0000","last_active":"2024-04-25T01:27:28.854644+0000","last_peered":"2024-04-25T01:27:28.854644+0000","last_clean":"2024-04-25T01:27:28.854644+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:27:28.854644+0000","last_undegraded":"2024-04-25T01:27:28.854644+0000","last_fullsized":"2024-04-25T01:27:28.854644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:51.821732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2519","reported_seq":2563,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:33.345264+0000","last_change":"2024-04-25T01:27:33.345287+0000","last_active":"2024-04-25T01:27:33.345264+0000","last_peered":"2024-04-25T01:27:33.345264+0000","last_clean":"2024-04-25T01:27:33.345264+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:27:33.345264+0000","last_undegraded":"2024-04-25T01:27:33.345264+0000","last_fullsized":"2024-04-25T01:27:33.345264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2519","last_scrub_stamp":"2024-04-25T01:27:33.345187+0000","last_deep_scrub":"27'2519","last_deep_scrub_stamp":"2024-04-25T01:27:33.345187+0000","last_clean_scrub_stamp":"2024-04-25T01:27:33.345187+0000","objects_scrubbed":0,"log_size":2519,"log_dups_size":0,"ondisk_log_size":2519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:19.300716+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2560","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.613797+0000","last_change":"2024-04-25T01:01:33.413401+0000","last_active":"2024-04-25T01:27:28.613797+0000","last_peered":"2024-04-25T01:27:28.613797+0000","last_clean":"2024-04-25T01:27:28.613797+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:27:28.613797+0000","last_undegraded":"2024-04-25T01:27:28.613797+0000","last_fullsized":"2024-04-25T01:27:28.613797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2560,"log_dups_size":0,"ondisk_log_size":2560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2589","reported_seq":2624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.032032+0000","last_change":"2024-04-25T01:01:33.404355+0000","last_active":"2024-04-25T01:27:29.032032+0000","last_peered":"2024-04-25T01:27:29.032032+0000","last_clean":"2024-04-25T01:27:29.032032+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:27:29.032032+0000","last_undegraded":"2024-04-25T01:27:29.032032+0000","last_fullsized":"2024-04-25T01:27:29.032032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:07.472693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2586","reported_seq":2622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.165039+0000","last_change":"2024-04-25T01:01:33.403471+0000","last_active":"2024-04-25T01:27:29.165039+0000","last_peered":"2024-04-25T01:27:29.165039+0000","last_clean":"2024-04-25T01:27:29.165039+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:27:29.165039+0000","last_undegraded":"2024-04-25T01:27:29.165039+0000","last_fullsized":"2024-04-25T01:27:29.165039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2586,"log_dups_size":0,"ondisk_log_size":2586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2585","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.726662+0000","last_change":"2024-04-25T01:01:33.396033+0000","last_active":"2024-04-25T01:27:28.726662+0000","last_peered":"2024-04-25T01:27:28.726662+0000","last_clean":"2024-04-25T01:27:28.726662+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:27:28.726662+0000","last_undegraded":"2024-04-25T01:27:28.726662+0000","last_fullsized":"2024-04-25T01:27:28.726662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.988531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2524","reported_seq":2559,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.844186+0000","last_change":"2024-04-25T01:01:33.400901+0000","last_active":"2024-04-25T01:27:28.844186+0000","last_peered":"2024-04-25T01:27:28.844186+0000","last_clean":"2024-04-25T01:27:28.844186+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:27:28.844186+0000","last_undegraded":"2024-04-25T01:27:28.844186+0000","last_fullsized":"2024-04-25T01:27:28.844186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:10.037639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2594","reported_seq":2629,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.090318+0000","last_change":"2024-04-25T01:01:33.410147+0000","last_active":"2024-04-25T01:27:29.090318+0000","last_peered":"2024-04-25T01:27:29.090318+0000","last_clean":"2024-04-25T01:27:29.090318+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:27:29.090318+0000","last_undegraded":"2024-04-25T01:27:29.090318+0000","last_fullsized":"2024-04-25T01:27:29.090318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2594,"log_dups_size":0,"ondisk_log_size":2594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2516","reported_seq":2551,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.536987+0000","last_change":"2024-04-25T01:01:33.403207+0000","last_active":"2024-04-25T01:27:29.536987+0000","last_peered":"2024-04-25T01:27:29.536987+0000","last_clean":"2024-04-25T01:27:29.536987+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:27:29.536987+0000","last_undegraded":"2024-04-25T01:27:29.536987+0000","last_fullsized":"2024-04-25T01:27:29.536987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2516,"log_dups_size":0,"ondisk_log_size":2516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.202229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2645","reported_seq":2681,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.459964+0000","last_change":"2024-04-25T01:01:33.388793+0000","last_active":"2024-04-25T01:27:29.459964+0000","last_peered":"2024-04-25T01:27:29.459964+0000","last_clean":"2024-04-25T01:27:29.459964+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:27:29.459964+0000","last_undegraded":"2024-04-25T01:27:29.459964+0000","last_fullsized":"2024-04-25T01:27:29.459964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2645,"log_dups_size":0,"ondisk_log_size":2645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2495","reported_seq":2530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.212978+0000","last_change":"2024-04-25T01:01:33.399156+0000","last_active":"2024-04-25T01:27:29.212978+0000","last_peered":"2024-04-25T01:27:29.212978+0000","last_clean":"2024-04-25T01:27:29.212978+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:27:29.212978+0000","last_undegraded":"2024-04-25T01:27:29.212978+0000","last_fullsized":"2024-04-25T01:27:29.212978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2495,"log_dups_size":0,"ondisk_log_size":2495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:21:24.306640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2489","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.402567+0000","last_change":"2024-04-25T01:27:34.402590+0000","last_active":"2024-04-25T01:27:34.402567+0000","last_peered":"2024-04-25T01:27:34.402567+0000","last_clean":"2024-04-25T01:27:34.402567+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:27:34.402567+0000","last_undegraded":"2024-04-25T01:27:34.402567+0000","last_fullsized":"2024-04-25T01:27:34.402567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2489","last_scrub_stamp":"2024-04-25T01:27:34.402474+0000","last_deep_scrub":"27'2489","last_deep_scrub_stamp":"2024-04-25T01:27:34.402474+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.402474+0000","objects_scrubbed":0,"log_size":2489,"log_dups_size":0,"ondisk_log_size":2489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:24.045721+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2444","reported_seq":2478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.135159+0000","last_change":"2024-04-25T01:01:33.406695+0000","last_active":"2024-04-25T01:27:29.135159+0000","last_peered":"2024-04-25T01:27:29.135159+0000","last_clean":"2024-04-25T01:27:29.135159+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:27:29.135159+0000","last_undegraded":"2024-04-25T01:27:29.135159+0000","last_fullsized":"2024-04-25T01:27:29.135159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:23:00.968769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2476","reported_seq":2516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.766420+0000","last_change":"2024-04-25T01:02:13.749183+0000","last_active":"2024-04-25T01:27:28.766420+0000","last_peered":"2024-04-25T01:27:28.766420+0000","last_clean":"2024-04-25T01:27:28.766420+0000","last_became_active":"2024-04-25T01:02:13.748901+0000","last_became_peered":"2024-04-25T01:02:13.748901+0000","last_unstale":"2024-04-25T01:27:28.766420+0000","last_undegraded":"2024-04-25T01:27:28.766420+0000","last_fullsized":"2024-04-25T01:27:28.766420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2612","reported_seq":2652,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.356550+0000","last_change":"2024-04-25T01:02:13.750135+0000","last_active":"2024-04-25T01:27:29.356550+0000","last_peered":"2024-04-25T01:27:29.356550+0000","last_clean":"2024-04-25T01:27:29.356550+0000","last_became_active":"2024-04-25T01:02:13.749859+0000","last_became_peered":"2024-04-25T01:02:13.749859+0000","last_unstale":"2024-04-25T01:27:29.356550+0000","last_undegraded":"2024-04-25T01:27:29.356550+0000","last_fullsized":"2024-04-25T01:27:29.356550+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2575","reported_seq":2609,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.586190+0000","last_change":"2024-04-25T01:01:33.411639+0000","last_active":"2024-04-25T01:27:29.586190+0000","last_peered":"2024-04-25T01:27:29.586190+0000","last_clean":"2024-04-25T01:27:29.586190+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:27:29.586190+0000","last_undegraded":"2024-04-25T01:27:29.586190+0000","last_fullsized":"2024-04-25T01:27:29.586190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:52.487669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2520","reported_seq":2541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.434424+0000","last_change":"2024-04-25T01:02:13.828931+0000","last_active":"2024-04-25T01:27:29.434424+0000","last_peered":"2024-04-25T01:27:29.434424+0000","last_clean":"2024-04-25T01:27:29.434424+0000","last_became_active":"2024-04-25T01:02:13.828599+0000","last_became_peered":"2024-04-25T01:02:13.828599+0000","last_unstale":"2024-04-25T01:27:29.434424+0000","last_undegraded":"2024-04-25T01:27:29.434424+0000","last_fullsized":"2024-04-25T01:27:29.434424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.39","version":"27'2527","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.026795+0000","last_change":"2024-04-25T01:01:33.392943+0000","last_active":"2024-04-25T01:27:29.026795+0000","last_peered":"2024-04-25T01:27:29.026795+0000","last_clean":"2024-04-25T01:27:29.026795+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:27:29.026795+0000","last_undegraded":"2024-04-25T01:27:29.026795+0000","last_fullsized":"2024-04-25T01:27:29.026795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2472","reported_seq":2506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.837234+0000","last_change":"2024-04-25T01:01:33.412800+0000","last_active":"2024-04-25T01:27:28.837234+0000","last_peered":"2024-04-25T01:27:28.837234+0000","last_clean":"2024-04-25T01:27:28.837234+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:27:28.837234+0000","last_undegraded":"2024-04-25T01:27:28.837234+0000","last_fullsized":"2024-04-25T01:27:28.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:06.408375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2510","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.623944+0000","last_change":"2024-04-25T01:01:33.395017+0000","last_active":"2024-04-25T01:27:29.623944+0000","last_peered":"2024-04-25T01:27:29.623944+0000","last_clean":"2024-04-25T01:27:29.623944+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:27:29.623944+0000","last_undegraded":"2024-04-25T01:27:29.623944+0000","last_fullsized":"2024-04-25T01:27:29.623944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:35.043313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2563","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.951522+0000","last_change":"2024-04-25T01:01:33.412452+0000","last_active":"2024-04-25T01:27:28.951522+0000","last_peered":"2024-04-25T01:27:28.951522+0000","last_clean":"2024-04-25T01:27:28.951522+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:27:28.951522+0000","last_undegraded":"2024-04-25T01:27:28.951522+0000","last_fullsized":"2024-04-25T01:27:28.951522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2563,"log_dups_size":0,"ondisk_log_size":2563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.199429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2630","reported_seq":2666,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.561637+0000","last_change":"2024-04-25T01:01:33.394296+0000","last_active":"2024-04-25T01:27:29.561637+0000","last_peered":"2024-04-25T01:27:29.561637+0000","last_clean":"2024-04-25T01:27:29.561637+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:27:29.561637+0000","last_undegraded":"2024-04-25T01:27:29.561637+0000","last_fullsized":"2024-04-25T01:27:29.561637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2630,"log_dups_size":0,"ondisk_log_size":2630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:26.684296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2469","reported_seq":2506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:28.732377+0000","last_change":"2024-04-25T01:01:33.398804+0000","last_active":"2024-04-25T01:27:28.732377+0000","last_peered":"2024-04-25T01:27:28.732377+0000","last_clean":"2024-04-25T01:27:28.732377+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:27:28.732377+0000","last_undegraded":"2024-04-25T01:27:28.732377+0000","last_fullsized":"2024-04-25T01:27:28.732377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2444","reported_seq":2480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:29.426402+0000","last_change":"2024-04-25T01:01:33.399365+0000","last_active":"2024-04-25T01:27:29.426402+0000","last_peered":"2024-04-25T01:27:29.426402+0000","last_clean":"2024-04-25T01:27:29.426402+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:27:29.426402+0000","last_undegraded":"2024-04-25T01:27:29.426402+0000","last_fullsized":"2024-04-25T01:27:29.426402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:01:32.374330+0000","last_clean_scrub_stamp":"2024-04-25T01:01:32.374330+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.444346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12020,"num_read_kb":3053080,"num_write":212634,"num_write_kb":30684192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232047,"ondisk_log_size":232047,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162605,"ondisk_log_size":162605,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":230494917,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":3112,"num_read_kb":19119,"num_write":209299,"num_write_kb":1676226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":461053952,"data_stored":460989834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194178,"ondisk_log_size":194178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575166,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":948336,"kb_used_data":46696,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93423504,"statfs":{"total":96636764160,"available":95665668096,"internally_reserved":0,"allocated":47816704,"data_stored":47635630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":923269578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575163,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180336,"kb_used_data":66600,"kb_used_omap":1,"kb_used_meta":1113726,"kb_avail":93191504,"statfs":{"total":96636764160,"available":95428100096,"internally_reserved":0,"allocated":68198400,"data_stored":68016872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1140455878},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":991788,"kb_used_data":40292,"kb_used_omap":1,"kb_used_meta":951486,"kb_avail":93380052,"statfs":{"total":96636764160,"available":95621173248,"internally_reserved":0,"allocated":41259008,"data_stored":41080585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":974322117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575163,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1003440,"kb_used_data":65832,"kb_used_omap":1,"kb_used_meta":937598,"kb_avail":93368400,"statfs":{"total":96636764160,"available":95609241600,"internally_reserved":0,"allocated":67411968,"data_stored":67228022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960100805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575166,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1125612,"kb_used_data":65828,"kb_used_omap":1,"kb_used_meta":1059774,"kb_avail":93246228,"statfs":{"total":96636764160,"available":95484137472,"internally_reserved":0,"allocated":67407872,"data_stored":67228568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1085209031},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575165,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676068,"kb_used_data":53532,"kb_used_omap":1,"kb_used_meta":622526,"kb_avail":93695772,"statfs":{"total":96636764160,"available":95944470528,"internally_reserved":0,"allocated":54816768,"data_stored":54644984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637467077},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575165,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1237104,"kb_used_data":60776,"kb_used_omap":1,"kb_used_meta":1176318,"kb_avail":93134736,"statfs":{"total":96636764160,"available":95369969664,"internally_reserved":0,"allocated":62234624,"data_stored":62052175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1204550085},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575165,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820980,"kb_used_data":54124,"kb_used_omap":1,"kb_used_meta":766846,"kb_avail":93550860,"statfs":{"total":96636764160,"available":95796080640,"internally_reserved":0,"allocated":55422976,"data_stored":55235990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":785250757},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54538240,"data_stored":54526590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61943808,"data_stored":61933143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54525952,"data_stored":54525952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40968192,"data_stored":40961553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":67907584,"data_stored":67897840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":46931968,"data_stored":46926230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:27:35.331 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.331 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.331 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.331 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.331 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.332 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.333 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.334 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.335 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T01:01:21.321421+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=21, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.336 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.337 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.338 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.339 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.340 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T01:01:31.368719+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=31, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.341 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-25T01:01:36.404982+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=36, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.342 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.343 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.344 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.345 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.345 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.345 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T01:01:32.374330+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=1, tm_sec=32, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=27, tm_sec=30, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:27:35.345 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T01:27:55.346 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:27:55.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.466+0000 7f30752b3640 1 Processor -- start 2024-04-25T01:27:55.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.466+0000 7f30752b3640 1 -- start start 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f30752b3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3070152fa0 0x7f30700ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f30752b3640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30700c9860 con 0x7f30701561e0 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f30752b3640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30700c5a70 con 0x7f30700c5520 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f30752b3640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f30700c5d50 con 0x7f3070152fa0 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f306f7fe640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f30701561e0 0x7f30700c8cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:53500/0 (socket says 172.21.15.129:53500) 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f306f7fe640 1 -- 172.21.15.129:0/2736032759 learned_addr learned my addr 172.21.15.129:0/2736032759 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f306effd640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f3070152fa0 0x7f30700ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f306effd640 1 -- 172.21.15.129:0/2736032759 >> v1:172.21.15.171:6789/0 conn(0x7f30700c5520 legacy=0x7f30700c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f306effd640 1 -- 172.21.15.129:0/2736032759 >> v1:172.21.15.129:6789/0 conn(0x7f30701561e0 legacy=0x7f30700c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.467+0000 7f306effd640 1 -- 172.21.15.129:0/2736032759 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30700c7650 con 0x7f3070152fa0 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.468+0000 7f306effd640 1 --2- 172.21.15.129:0/2736032759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3070152fa0 0x7f30700ca2e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f30640097a0 tx=0x7f306402a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751a5c8dd6fd2b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.468+0000 7f306dffb640 1 -- 172.21.15.129:0/2736032759 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064037070 con 0x7f3070152fa0 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.468+0000 7f306dffb640 1 -- 172.21.15.129:0/2736032759 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306402ac90 con 0x7f3070152fa0 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.468+0000 7f306dffb640 1 -- 172.21.15.129:0/2736032759 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064032660 con 0x7f3070152fa0 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 -- 172.21.15.129:0/2736032759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3070152fa0 msgr2=0x7f30700ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 --2- 172.21.15.129:0/2736032759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3070152fa0 0x7f30700ca2e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f30640097a0 tx=0x7f306402a000 comp rx=0 tx=0).stop 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 -- 172.21.15.129:0/2736032759 shutdown_connections 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 --2- 172.21.15.129:0/2736032759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3070152fa0 0x7f30700ca2e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 -- 172.21.15.129:0/2736032759 >> 172.21.15.129:0/2736032759 conn(0x7f307006c400 msgr2=0x7f307006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 -- 172.21.15.129:0/2736032759 shutdown_connections 2024-04-25T01:27:55.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.469+0000 7f30752b3640 1 -- 172.21.15.129:0/2736032759 wait complete. 2024-04-25T01:27:55.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.470+0000 7f30752b3640 1 Processor -- start 2024-04-25T01:27:55.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.470+0000 7f30752b3640 1 -- start start 2024-04-25T01:27:55.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 0x7f30700c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3070152fa0 0x7f307014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30701561e0 0x7f307014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30700c93d0 con 0x7f30700c5520 2024-04-25T01:27:55.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f30700c6030 con 0x7f30701561e0 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f30700c9550 con 0x7f3070152fa0 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 0x7f30700c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 0x7f30700c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56916/0 (socket says 172.21.15.129:56916) 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 -- 172.21.15.129:0/763469354 learned_addr learned my addr 172.21.15.129:0/763469354 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306f7fe640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30701561e0 0x7f307014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306e7fc640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3070152fa0 0x7f307014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 -- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3070152fa0 msgr2=0x7f307014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3070152fa0 0x7f307014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 -- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30701561e0 msgr2=0x7f307014beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30701561e0 0x7f307014beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 -- 172.21.15.129:0/763469354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307014fd70 con 0x7f30700c5520 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f306effd640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 0x7f30700c42f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f30640098c0 tx=0x7f30640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064037040 con 0x7f30700c5520 2024-04-25T01:27:55.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064004230 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30700c1d10 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.471+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064003610 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.472+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30701ec6a0 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.473+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3070075550 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.473+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30640037b0 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.473+0000 7f30577fe640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f305c06be30 0x7f305c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.473+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f30640abc50 con 0x7f30700c5520 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.474+0000 7f306e7fc640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f305c06be30 0x7f305c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.454 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.474+0000 7f306e7fc640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f305c06be30 0x7f305c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f30700be690 tx=0x7f3060009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:55.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.478+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f30640b1020 con 0x7f30700c5520 2024-04-25T01:27:55.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.655+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 --> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30700ca0c0 con 0x7f305c06be30 2024-04-25T01:27:55.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.663+0000 7f30577fe640 1 -- 172.21.15.129:0/763469354 <== mgr.4105 v2:172.21.15.171:6832/1668644098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336689 (secure 0 0 0) 0x7f30700ca0c0 con 0x7f305c06be30 2024-04-25T01:27:55.645 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:55.647 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.666+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f305c06be30 msgr2=0x7f305c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.666+0000 7f30752b3640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f305c06be30 0x7f305c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f30700be690 tx=0x7f3060009040 comp rx=0 tx=0).stop 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.666+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 msgr2=0x7f30700c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.666+0000 7f30752b3640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 0x7f30700c42f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f30640098c0 tx=0x7f30640046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 shutdown_connections 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f305c06be30 0x7f305c06e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3070152fa0 0x7f307014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30701561e0 0x7f307014beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 --2- 172.21.15.129:0/763469354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30700c5520 0x7f30700c42f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 >> 172.21.15.129:0/763469354 conn(0x7f307006c400 msgr2=0x7f30700c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 shutdown_connections 2024-04-25T01:27:55.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.667+0000 7f30752b3640 1 -- 172.21.15.129:0/763469354 wait complete. 2024-04-25T01:27:55.656 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":818,"stamp":"2024-04-25T01:27:55.063456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231086423,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":15248,"num_read_kb":3072299,"num_write":423214,"num_write_kb":32362837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5927,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589018,"ondisk_log_size":589018,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":7983664,"kb_used_data":453680,"kb_used_omap":12,"kb_used_meta":7529907,"kb_avail":746991056,"statfs":{"total":773094113280,"available":764918841344,"internally_reserved":0,"allocated":464568320,"data_stored":463125102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":7710625328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":366,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002634"},"pg_stats":[{"pgid":"2.1f","version":"27'2278","reported_seq":2325,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.587916+0000","last_change":"2024-04-25T01:27:34.587961+0000","last_active":"2024-04-25T01:27:34.587916+0000","last_peered":"2024-04-25T01:27:34.587916+0000","last_clean":"2024-04-25T01:27:34.587916+0000","last_became_active":"2024-04-25T01:01:32.398934+0000","last_became_peered":"2024-04-25T01:01:32.398934+0000","last_unstale":"2024-04-25T01:27:34.587916+0000","last_undegraded":"2024-04-25T01:27:34.587916+0000","last_fullsized":"2024-04-25T01:27:34.587916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2278","last_scrub_stamp":"2024-04-25T01:27:34.587793+0000","last_deep_scrub":"27'2278","last_deep_scrub_stamp":"2024-04-25T01:27:34.587793+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.587793+0000","objects_scrubbed":2,"log_size":2278,"log_dups_size":0,"ondisk_log_size":2278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:20.543322+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2285,"num_write_kb":17945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":561,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2549","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:42.388949+0000","last_change":"2024-04-25T01:27:42.388949+0000","last_active":"2024-04-25T01:27:42.388949+0000","last_peered":"2024-04-25T01:27:42.388949+0000","last_clean":"2024-04-25T01:27:42.388949+0000","last_became_active":"2024-04-25T01:01:33.400437+0000","last_became_peered":"2024-04-25T01:01:33.400437+0000","last_unstale":"2024-04-25T01:27:42.388949+0000","last_undegraded":"2024-04-25T01:27:42.388949+0000","last_fullsized":"2024-04-25T01:27:42.388949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2549","last_scrub_stamp":"2024-04-25T01:27:42.388823+0000","last_deep_scrub":"27'2549","last_deep_scrub_stamp":"2024-04-25T01:27:42.388823+0000","last_clean_scrub_stamp":"2024-04-25T01:27:42.388823+0000","objects_scrubbed":0,"log_size":2549,"log_dups_size":0,"ondisk_log_size":2549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:22.709995+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9461","reported_seq":9788,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:50.477356+0000","last_change":"2024-04-25T01:27:50.477381+0000","last_active":"2024-04-25T01:27:50.477356+0000","last_peered":"2024-04-25T01:27:50.477356+0000","last_clean":"2024-04-25T01:27:50.477356+0000","last_became_active":"2024-04-25T01:01:37.440753+0000","last_became_peered":"2024-04-25T01:01:37.440753+0000","last_unstale":"2024-04-25T01:27:50.477356+0000","last_undegraded":"2024-04-25T01:27:50.477356+0000","last_fullsized":"2024-04-25T01:27:50.477356+0000","mapping_epoch":23,"log_start":"27'3202","ondisk_log_start":"27'3202","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9461","last_scrub_stamp":"2024-04-25T01:27:50.477271+0000","last_deep_scrub":"27'9461","last_deep_scrub_stamp":"2024-04-25T01:27:50.477271+0000","last_clean_scrub_stamp":"2024-04-25T01:27:50.477271+0000","objects_scrubbed":0,"log_size":6259,"log_dups_size":0,"ondisk_log_size":6259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.070103+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":290,"num_read_kb":73660,"num_write":6825,"num_write_kb":1007677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6106","reported_seq":6154,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.358698+0000","last_change":"2024-04-25T01:27:38.358698+0000","last_active":"2024-04-25T01:27:38.358698+0000","last_peered":"2024-04-25T01:27:38.358698+0000","last_clean":"2024-04-25T01:27:38.358698+0000","last_became_active":"2024-04-25T01:01:32.395186+0000","last_became_peered":"2024-04-25T01:01:32.395186+0000","last_unstale":"2024-04-25T01:27:38.358698+0000","last_undegraded":"2024-04-25T01:27:38.358698+0000","last_fullsized":"2024-04-25T01:27:38.358698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6106","last_scrub_stamp":"2024-04-25T01:27:38.358559+0000","last_deep_scrub":"27'6106","last_deep_scrub_stamp":"2024-04-25T01:27:38.358559+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.358559+0000","objects_scrubbed":0,"log_size":6106,"log_dups_size":0,"ondisk_log_size":6106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:40.538257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":6114,"num_write_kb":49190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2532","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.691095+0000","last_change":"2024-04-25T01:27:43.691135+0000","last_active":"2024-04-25T01:27:43.691095+0000","last_peered":"2024-04-25T01:27:43.691095+0000","last_clean":"2024-04-25T01:27:43.691095+0000","last_became_active":"2024-04-25T01:01:33.409470+0000","last_became_peered":"2024-04-25T01:01:33.409470+0000","last_unstale":"2024-04-25T01:27:43.691095+0000","last_undegraded":"2024-04-25T01:27:43.691095+0000","last_fullsized":"2024-04-25T01:27:43.691095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2532","last_scrub_stamp":"2024-04-25T01:27:43.691000+0000","last_deep_scrub":"27'2532","last_deep_scrub_stamp":"2024-04-25T01:27:43.691000+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.691000+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:02.968659+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9580","reported_seq":10046,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.784891+0000","last_change":"2024-04-25T01:27:41.784915+0000","last_active":"2024-04-25T01:27:41.784891+0000","last_peered":"2024-04-25T01:27:41.784891+0000","last_clean":"2024-04-25T01:27:41.784891+0000","last_became_active":"2024-04-25T01:01:37.434996+0000","last_became_peered":"2024-04-25T01:01:37.434996+0000","last_unstale":"2024-04-25T01:27:41.784891+0000","last_undegraded":"2024-04-25T01:27:41.784891+0000","last_fullsized":"2024-04-25T01:27:41.784891+0000","mapping_epoch":23,"log_start":"27'2431","ondisk_log_start":"27'2431","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9580","last_scrub_stamp":"2024-04-25T01:27:41.784810+0000","last_deep_scrub":"27'9580","last_deep_scrub_stamp":"2024-04-25T01:27:41.784810+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.784810+0000","objects_scrubbed":0,"log_size":7149,"log_dups_size":0,"ondisk_log_size":7149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:59.489502+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":429,"num_read_kb":108966,"num_write":6832,"num_write_kb":1087543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9181","reported_seq":9319,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.365051+0000","last_change":"2024-04-25T01:27:37.365081+0000","last_active":"2024-04-25T01:27:37.365051+0000","last_peered":"2024-04-25T01:27:37.365051+0000","last_clean":"2024-04-25T01:27:37.365051+0000","last_became_active":"2024-04-25T01:01:32.397956+0000","last_became_peered":"2024-04-25T01:01:32.397956+0000","last_unstale":"2024-04-25T01:27:37.365051+0000","last_undegraded":"2024-04-25T01:27:37.365051+0000","last_fullsized":"2024-04-25T01:27:37.365051+0000","mapping_epoch":18,"log_start":"27'2200","ondisk_log_start":"27'2200","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9181","last_scrub_stamp":"2024-04-25T01:27:37.364988+0000","last_deep_scrub":"27'9181","last_deep_scrub_stamp":"2024-04-25T01:27:37.364988+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.364988+0000","objects_scrubbed":3,"log_size":6981,"log_dups_size":0,"ondisk_log_size":6981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:08.912742+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4982608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":93,"num_read_kb":825,"num_write":9172,"num_write_kb":76484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2632","reported_seq":2675,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.453113+0000","last_change":"2024-04-25T01:27:35.453133+0000","last_active":"2024-04-25T01:27:35.453113+0000","last_peered":"2024-04-25T01:27:35.453113+0000","last_clean":"2024-04-25T01:27:35.453113+0000","last_became_active":"2024-04-25T01:01:33.410935+0000","last_became_peered":"2024-04-25T01:01:33.410935+0000","last_unstale":"2024-04-25T01:27:35.453113+0000","last_undegraded":"2024-04-25T01:27:35.453113+0000","last_fullsized":"2024-04-25T01:27:35.453113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2632","last_scrub_stamp":"2024-04-25T01:27:35.453051+0000","last_deep_scrub":"27'2632","last_deep_scrub_stamp":"2024-04-25T01:27:35.453051+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.453051+0000","objects_scrubbed":0,"log_size":2632,"log_dups_size":0,"ondisk_log_size":2632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:07.000805+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9168","reported_seq":9794,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.722288+0000","last_change":"2024-04-25T01:27:43.722313+0000","last_active":"2024-04-25T01:27:43.722288+0000","last_peered":"2024-04-25T01:27:43.722288+0000","last_clean":"2024-04-25T01:27:43.722288+0000","last_became_active":"2024-04-25T01:01:37.436800+0000","last_became_peered":"2024-04-25T01:01:37.436800+0000","last_unstale":"2024-04-25T01:27:43.722288+0000","last_undegraded":"2024-04-25T01:27:43.722288+0000","last_fullsized":"2024-04-25T01:27:43.722288+0000","mapping_epoch":23,"log_start":"27'2014","ondisk_log_start":"27'2014","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9168","last_scrub_stamp":"2024-04-25T01:27:43.722206+0000","last_deep_scrub":"27'9168","last_deep_scrub_stamp":"2024-04-25T01:27:43.722206+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.722206+0000","objects_scrubbed":0,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:43.614706+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6458,"num_write_kb":913780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4332","reported_seq":4377,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.293275+0000","last_change":"2024-04-25T01:27:41.293298+0000","last_active":"2024-04-25T01:27:41.293275+0000","last_peered":"2024-04-25T01:27:41.293275+0000","last_clean":"2024-04-25T01:27:41.293275+0000","last_became_active":"2024-04-25T01:01:32.399584+0000","last_became_peered":"2024-04-25T01:01:32.399584+0000","last_unstale":"2024-04-25T01:27:41.293275+0000","last_undegraded":"2024-04-25T01:27:41.293275+0000","last_fullsized":"2024-04-25T01:27:41.293275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4332","last_scrub_stamp":"2024-04-25T01:27:41.293212+0000","last_deep_scrub":"27'4332","last_deep_scrub_stamp":"2024-04-25T01:27:41.293212+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.293212+0000","objects_scrubbed":1,"log_size":4332,"log_dups_size":0,"ondisk_log_size":4332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:34.540353+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4333,"num_write_kb":43749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2569","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.498636+0000","last_change":"2024-04-25T01:27:34.498658+0000","last_active":"2024-04-25T01:27:34.498636+0000","last_peered":"2024-04-25T01:27:34.498636+0000","last_clean":"2024-04-25T01:27:34.498636+0000","last_became_active":"2024-04-25T01:01:33.410164+0000","last_became_peered":"2024-04-25T01:01:33.410164+0000","last_unstale":"2024-04-25T01:27:34.498636+0000","last_undegraded":"2024-04-25T01:27:34.498636+0000","last_fullsized":"2024-04-25T01:27:34.498636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2569","last_scrub_stamp":"2024-04-25T01:27:34.498558+0000","last_deep_scrub":"27'2569","last_deep_scrub_stamp":"2024-04-25T01:27:34.498558+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.498558+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:42.151706+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'8842","reported_seq":9262,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.807511+0000","last_change":"2024-04-25T01:27:40.807538+0000","last_active":"2024-04-25T01:27:40.807511+0000","last_peered":"2024-04-25T01:27:40.807511+0000","last_clean":"2024-04-25T01:27:40.807511+0000","last_became_active":"2024-04-25T01:01:37.430736+0000","last_became_peered":"2024-04-25T01:01:37.430736+0000","last_unstale":"2024-04-25T01:27:40.807511+0000","last_undegraded":"2024-04-25T01:27:40.807511+0000","last_fullsized":"2024-04-25T01:27:40.807511+0000","mapping_epoch":23,"log_start":"27'1610","ondisk_log_start":"27'1610","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8842","last_scrub_stamp":"2024-04-25T01:27:40.807423+0000","last_deep_scrub":"27'8842","last_deep_scrub_stamp":"2024-04-25T01:27:40.807423+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.807423+0000","objects_scrubbed":0,"log_size":7232,"log_dups_size":0,"ondisk_log_size":7232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:13.665309+0000","scrub_duration":50,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":383,"num_read_kb":97282,"num_write":6140,"num_write_kb":837273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2555","reported_seq":2600,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.983823+0000","last_change":"2024-04-25T01:27:39.983823+0000","last_active":"2024-04-25T01:27:39.983823+0000","last_peered":"2024-04-25T01:27:39.983823+0000","last_clean":"2024-04-25T01:27:39.983823+0000","last_became_active":"2024-04-25T01:01:33.400637+0000","last_became_peered":"2024-04-25T01:01:33.400637+0000","last_unstale":"2024-04-25T01:27:39.983823+0000","last_undegraded":"2024-04-25T01:27:39.983823+0000","last_fullsized":"2024-04-25T01:27:39.983823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2555","last_scrub_stamp":"2024-04-25T01:27:39.983704+0000","last_deep_scrub":"27'2555","last_deep_scrub_stamp":"2024-04-25T01:27:39.983704+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.983704+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:42.299263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'7003","reported_seq":7049,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:42.316907+0000","last_change":"2024-04-25T01:27:42.316947+0000","last_active":"2024-04-25T01:27:42.316907+0000","last_peered":"2024-04-25T01:27:42.316907+0000","last_clean":"2024-04-25T01:27:42.316907+0000","last_became_active":"2024-04-25T01:01:32.400602+0000","last_became_peered":"2024-04-25T01:01:32.400602+0000","last_unstale":"2024-04-25T01:27:42.316907+0000","last_undegraded":"2024-04-25T01:27:42.316907+0000","last_fullsized":"2024-04-25T01:27:42.316907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7003","last_scrub_stamp":"2024-04-25T01:27:42.316841+0000","last_deep_scrub":"27'7003","last_deep_scrub_stamp":"2024-04-25T01:27:42.316841+0000","last_clean_scrub_stamp":"2024-04-25T01:27:42.316841+0000","objects_scrubbed":4,"log_size":7003,"log_dups_size":0,"ondisk_log_size":7003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:33.842327+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6999,"num_write_kb":53860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":366,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9960","reported_seq":10591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:44.740038+0000","last_change":"2024-04-25T01:27:44.740065+0000","last_active":"2024-04-25T01:27:44.740038+0000","last_peered":"2024-04-25T01:27:44.740038+0000","last_clean":"2024-04-25T01:27:44.740038+0000","last_became_active":"2024-04-25T01:01:37.436738+0000","last_became_peered":"2024-04-25T01:01:37.436738+0000","last_unstale":"2024-04-25T01:27:44.740038+0000","last_undegraded":"2024-04-25T01:27:44.740038+0000","last_fullsized":"2024-04-25T01:27:44.740038+0000","mapping_epoch":23,"log_start":"27'2730","ondisk_log_start":"27'2730","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9960","last_scrub_stamp":"2024-04-25T01:27:44.739939+0000","last_deep_scrub":"27'9960","last_deep_scrub_stamp":"2024-04-25T01:27:44.739939+0000","last_clean_scrub_stamp":"2024-04-25T01:27:44.739939+0000","objects_scrubbed":0,"log_size":7230,"log_dups_size":0,"ondisk_log_size":7230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:02.818296+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":593,"num_read_kb":150622,"num_write":7206,"num_write_kb":1017663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2576","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.380634+0000","last_change":"2024-04-25T01:27:43.380675+0000","last_active":"2024-04-25T01:27:43.380634+0000","last_peered":"2024-04-25T01:27:43.380634+0000","last_clean":"2024-04-25T01:27:43.380634+0000","last_became_active":"2024-04-25T01:01:33.401176+0000","last_became_peered":"2024-04-25T01:01:33.401176+0000","last_unstale":"2024-04-25T01:27:43.380634+0000","last_undegraded":"2024-04-25T01:27:43.380634+0000","last_fullsized":"2024-04-25T01:27:43.380634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2576","last_scrub_stamp":"2024-04-25T01:27:43.380519+0000","last_deep_scrub":"27'2576","last_deep_scrub_stamp":"2024-04-25T01:27:43.380519+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.380519+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:02.658177+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7177","reported_seq":7224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.062371+0000","last_change":"2024-04-25T01:27:37.062371+0000","last_active":"2024-04-25T01:27:37.062371+0000","last_peered":"2024-04-25T01:27:37.062371+0000","last_clean":"2024-04-25T01:27:37.062371+0000","last_became_active":"2024-04-25T01:01:32.396532+0000","last_became_peered":"2024-04-25T01:01:32.396532+0000","last_unstale":"2024-04-25T01:27:37.062371+0000","last_undegraded":"2024-04-25T01:27:37.062371+0000","last_fullsized":"2024-04-25T01:27:37.062371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7177","last_scrub_stamp":"2024-04-25T01:27:37.062273+0000","last_deep_scrub":"27'7177","last_deep_scrub_stamp":"2024-04-25T01:27:37.062273+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.062273+0000","objects_scrubbed":2,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:34.192879+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7178,"num_write_kb":58045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9188","reported_seq":9693,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.725013+0000","last_change":"2024-04-25T01:27:45.725057+0000","last_active":"2024-04-25T01:27:45.725013+0000","last_peered":"2024-04-25T01:27:45.725013+0000","last_clean":"2024-04-25T01:27:45.725013+0000","last_became_active":"2024-04-25T01:01:37.439784+0000","last_became_peered":"2024-04-25T01:01:37.439784+0000","last_unstale":"2024-04-25T01:27:45.725013+0000","last_undegraded":"2024-04-25T01:27:45.725013+0000","last_fullsized":"2024-04-25T01:27:45.725013+0000","mapping_epoch":23,"log_start":"27'2033","ondisk_log_start":"27'2033","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9188","last_scrub_stamp":"2024-04-25T01:27:45.724882+0000","last_deep_scrub":"27'9188","last_deep_scrub_stamp":"2024-04-25T01:27:45.724882+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.724882+0000","objects_scrubbed":0,"log_size":7155,"log_dups_size":0,"ondisk_log_size":7155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:33.738049+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":118618,"num_write":6486,"num_write_kb":966622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2640","reported_seq":2685,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.406042+0000","last_change":"2024-04-25T01:27:41.406064+0000","last_active":"2024-04-25T01:27:41.406042+0000","last_peered":"2024-04-25T01:27:41.406042+0000","last_clean":"2024-04-25T01:27:41.406042+0000","last_became_active":"2024-04-25T01:01:33.397416+0000","last_became_peered":"2024-04-25T01:01:33.397416+0000","last_unstale":"2024-04-25T01:27:41.406042+0000","last_undegraded":"2024-04-25T01:27:41.406042+0000","last_fullsized":"2024-04-25T01:27:41.406042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2640","last_scrub_stamp":"2024-04-25T01:27:41.405964+0000","last_deep_scrub":"27'2640","last_deep_scrub_stamp":"2024-04-25T01:27:41.405964+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.405964+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:37.618584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3548","reported_seq":3595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.019360+0000","last_change":"2024-04-25T01:27:39.019397+0000","last_active":"2024-04-25T01:27:39.019360+0000","last_peered":"2024-04-25T01:27:39.019360+0000","last_clean":"2024-04-25T01:27:39.019360+0000","last_became_active":"2024-04-25T01:01:32.402376+0000","last_became_peered":"2024-04-25T01:01:32.402376+0000","last_unstale":"2024-04-25T01:27:39.019360+0000","last_undegraded":"2024-04-25T01:27:39.019360+0000","last_fullsized":"2024-04-25T01:27:39.019360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3548","last_scrub_stamp":"2024-04-25T01:27:39.019296+0000","last_deep_scrub":"27'3548","last_deep_scrub_stamp":"2024-04-25T01:27:39.019296+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.019296+0000","objects_scrubbed":0,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:14.384689+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":26965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'8991","reported_seq":9516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.042105+0000","last_change":"2024-04-25T01:27:41.042132+0000","last_active":"2024-04-25T01:27:41.042105+0000","last_peered":"2024-04-25T01:27:41.042105+0000","last_clean":"2024-04-25T01:27:41.042105+0000","last_became_active":"2024-04-25T01:02:13.748037+0000","last_became_peered":"2024-04-25T01:02:13.748037+0000","last_unstale":"2024-04-25T01:27:41.042105+0000","last_undegraded":"2024-04-25T01:27:41.042105+0000","last_fullsized":"2024-04-25T01:27:41.042105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8991","last_scrub_stamp":"2024-04-25T01:27:41.042026+0000","last_deep_scrub":"27'8991","last_deep_scrub_stamp":"2024-04-25T01:27:41.042026+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.042026+0000","objects_scrubbed":0,"log_size":8991,"log_dups_size":0,"ondisk_log_size":8991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:04.260519+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":493,"num_read_kb":125222,"num_write":6426,"num_write_kb":893296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2544","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.736947+0000","last_change":"2024-04-25T01:27:37.736947+0000","last_active":"2024-04-25T01:27:37.736947+0000","last_peered":"2024-04-25T01:27:37.736947+0000","last_clean":"2024-04-25T01:27:37.736947+0000","last_became_active":"2024-04-25T01:01:33.401365+0000","last_became_peered":"2024-04-25T01:01:33.401365+0000","last_unstale":"2024-04-25T01:27:37.736947+0000","last_undegraded":"2024-04-25T01:27:37.736947+0000","last_fullsized":"2024-04-25T01:27:37.736947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2544","last_scrub_stamp":"2024-04-25T01:27:37.736838+0000","last_deep_scrub":"27'2544","last_deep_scrub_stamp":"2024-04-25T01:27:37.736838+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.736838+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:23.692367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2990","reported_seq":3037,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.314470+0000","last_change":"2024-04-25T01:27:38.314493+0000","last_active":"2024-04-25T01:27:38.314470+0000","last_peered":"2024-04-25T01:27:38.314470+0000","last_clean":"2024-04-25T01:27:38.314470+0000","last_became_active":"2024-04-25T01:01:32.398929+0000","last_became_peered":"2024-04-25T01:01:32.398929+0000","last_unstale":"2024-04-25T01:27:38.314470+0000","last_undegraded":"2024-04-25T01:27:38.314470+0000","last_fullsized":"2024-04-25T01:27:38.314470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2990","last_scrub_stamp":"2024-04-25T01:27:38.314411+0000","last_deep_scrub":"27'2990","last_deep_scrub_stamp":"2024-04-25T01:27:38.314411+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.314411+0000","objects_scrubbed":1,"log_size":2990,"log_dups_size":0,"ondisk_log_size":2990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:35.445016+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2993,"num_write_kb":22443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9534","reported_seq":9766,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:42.756546+0000","last_change":"2024-04-25T01:27:42.756571+0000","last_active":"2024-04-25T01:27:42.756546+0000","last_peered":"2024-04-25T01:27:42.756546+0000","last_clean":"2024-04-25T01:27:42.756546+0000","last_became_active":"2024-04-25T01:01:37.435734+0000","last_became_peered":"2024-04-25T01:01:37.435734+0000","last_unstale":"2024-04-25T01:27:42.756546+0000","last_undegraded":"2024-04-25T01:27:42.756546+0000","last_fullsized":"2024-04-25T01:27:42.756546+0000","mapping_epoch":23,"log_start":"27'2346","ondisk_log_start":"27'2346","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9534","last_scrub_stamp":"2024-04-25T01:27:42.756463+0000","last_deep_scrub":"27'9534","last_deep_scrub_stamp":"2024-04-25T01:27:42.756463+0000","last_clean_scrub_stamp":"2024-04-25T01:27:42.756463+0000","objects_scrubbed":0,"log_size":7188,"log_dups_size":0,"ondisk_log_size":7188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:47.388307+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":49022,"num_write":6898,"num_write_kb":954026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2508","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.699185+0000","last_change":"2024-04-25T01:27:38.699206+0000","last_active":"2024-04-25T01:27:38.699185+0000","last_peered":"2024-04-25T01:27:38.699185+0000","last_clean":"2024-04-25T01:27:38.699185+0000","last_became_active":"2024-04-25T01:01:33.398917+0000","last_became_peered":"2024-04-25T01:01:33.398917+0000","last_unstale":"2024-04-25T01:27:38.699185+0000","last_undegraded":"2024-04-25T01:27:38.699185+0000","last_fullsized":"2024-04-25T01:27:38.699185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2508","last_scrub_stamp":"2024-04-25T01:27:38.699103+0000","last_deep_scrub":"27'2508","last_deep_scrub_stamp":"2024-04-25T01:27:38.699103+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.699103+0000","objects_scrubbed":0,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:43.330946+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4628","reported_seq":4675,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.979391+0000","last_change":"2024-04-25T01:27:38.979442+0000","last_active":"2024-04-25T01:27:38.979391+0000","last_peered":"2024-04-25T01:27:38.979391+0000","last_clean":"2024-04-25T01:27:38.979391+0000","last_became_active":"2024-04-25T01:01:32.402390+0000","last_became_peered":"2024-04-25T01:01:32.402390+0000","last_unstale":"2024-04-25T01:27:38.979391+0000","last_undegraded":"2024-04-25T01:27:38.979391+0000","last_fullsized":"2024-04-25T01:27:38.979391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4628","last_scrub_stamp":"2024-04-25T01:27:38.979301+0000","last_deep_scrub":"27'4628","last_deep_scrub_stamp":"2024-04-25T01:27:38.979301+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.979301+0000","objects_scrubbed":4,"log_size":4628,"log_dups_size":0,"ondisk_log_size":4628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:28.622548+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4622,"num_write_kb":39981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9265","reported_seq":9341,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.815006+0000","last_change":"2024-04-25T01:27:39.815030+0000","last_active":"2024-04-25T01:27:39.815006+0000","last_peered":"2024-04-25T01:27:39.815006+0000","last_clean":"2024-04-25T01:27:39.815006+0000","last_became_active":"2024-04-25T01:01:37.429576+0000","last_became_peered":"2024-04-25T01:01:37.429576+0000","last_unstale":"2024-04-25T01:27:39.815006+0000","last_undegraded":"2024-04-25T01:27:39.815006+0000","last_fullsized":"2024-04-25T01:27:39.815006+0000","mapping_epoch":23,"log_start":"27'2105","ondisk_log_start":"27'2105","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9265","last_scrub_stamp":"2024-04-25T01:27:39.814911+0000","last_deep_scrub":"27'9265","last_deep_scrub_stamp":"2024-04-25T01:27:39.814911+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.814911+0000","objects_scrubbed":0,"log_size":7160,"log_dups_size":0,"ondisk_log_size":7160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:42.130471+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":9398,"num_write":6620,"num_write_kb":863134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2500","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:44.362259+0000","last_change":"2024-04-25T01:27:44.362299+0000","last_active":"2024-04-25T01:27:44.362259+0000","last_peered":"2024-04-25T01:27:44.362259+0000","last_clean":"2024-04-25T01:27:44.362259+0000","last_became_active":"2024-04-25T01:01:33.403667+0000","last_became_peered":"2024-04-25T01:01:33.403667+0000","last_unstale":"2024-04-25T01:27:44.362259+0000","last_undegraded":"2024-04-25T01:27:44.362259+0000","last_fullsized":"2024-04-25T01:27:44.362259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2500","last_scrub_stamp":"2024-04-25T01:27:44.362147+0000","last_deep_scrub":"27'2500","last_deep_scrub_stamp":"2024-04-25T01:27:44.362147+0000","last_clean_scrub_stamp":"2024-04-25T01:27:44.362147+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:22.191860+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6904","reported_seq":7614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.165588+0000","last_change":"2024-04-25T01:27:36.165616+0000","last_active":"2024-04-25T01:27:36.165588+0000","last_peered":"2024-04-25T01:27:36.165588+0000","last_clean":"2024-04-25T01:27:36.165588+0000","last_became_active":"2024-04-25T01:01:32.394350+0000","last_became_peered":"2024-04-25T01:01:32.394350+0000","last_unstale":"2024-04-25T01:27:36.165588+0000","last_undegraded":"2024-04-25T01:27:36.165588+0000","last_fullsized":"2024-04-25T01:27:36.165588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6904","last_scrub_stamp":"2024-04-25T01:27:36.165523+0000","last_deep_scrub":"27'6904","last_deep_scrub_stamp":"2024-04-25T01:27:36.165523+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.165523+0000","objects_scrubbed":3,"log_size":6904,"log_dups_size":0,"ondisk_log_size":6904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:22.121052+0000","scrub_duration":80,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":663,"num_read_kb":4517,"num_write":9705,"num_write_kb":61055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9005","reported_seq":9087,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.431567+0000","last_change":"2024-04-25T01:27:47.431567+0000","last_active":"2024-04-25T01:27:47.431567+0000","last_peered":"2024-04-25T01:27:47.431567+0000","last_clean":"2024-04-25T01:27:47.431567+0000","last_became_active":"2024-04-25T01:01:37.434210+0000","last_became_peered":"2024-04-25T01:01:37.434210+0000","last_unstale":"2024-04-25T01:27:47.431567+0000","last_undegraded":"2024-04-25T01:27:47.431567+0000","last_fullsized":"2024-04-25T01:27:47.431567+0000","mapping_epoch":23,"log_start":"27'2722","ondisk_log_start":"27'2722","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9005","last_scrub_stamp":"2024-04-25T01:27:47.431452+0000","last_deep_scrub":"27'9005","last_deep_scrub_stamp":"2024-04-25T01:27:47.431452+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.431452+0000","objects_scrubbed":0,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:32.497061+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":42,"num_read_kb":10668,"num_write":6336,"num_write_kb":893375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2596","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.990924+0000","last_change":"2024-04-25T01:27:43.990947+0000","last_active":"2024-04-25T01:27:43.990924+0000","last_peered":"2024-04-25T01:27:43.990924+0000","last_clean":"2024-04-25T01:27:43.990924+0000","last_became_active":"2024-04-25T01:01:33.399019+0000","last_became_peered":"2024-04-25T01:01:33.399019+0000","last_unstale":"2024-04-25T01:27:43.990924+0000","last_undegraded":"2024-04-25T01:27:43.990924+0000","last_fullsized":"2024-04-25T01:27:43.990924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2596","last_scrub_stamp":"2024-04-25T01:27:43.990817+0000","last_deep_scrub":"27'2596","last_deep_scrub_stamp":"2024-04-25T01:27:43.990817+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.990817+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:32.003983+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6066","reported_seq":6113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.278989+0000","last_change":"2024-04-25T01:27:39.279035+0000","last_active":"2024-04-25T01:27:39.278989+0000","last_peered":"2024-04-25T01:27:39.278989+0000","last_clean":"2024-04-25T01:27:39.278989+0000","last_became_active":"2024-04-25T01:01:32.399699+0000","last_became_peered":"2024-04-25T01:01:32.399699+0000","last_unstale":"2024-04-25T01:27:39.278989+0000","last_undegraded":"2024-04-25T01:27:39.278989+0000","last_fullsized":"2024-04-25T01:27:39.278989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6066","last_scrub_stamp":"2024-04-25T01:27:39.278882+0000","last_deep_scrub":"27'6066","last_deep_scrub_stamp":"2024-04-25T01:27:39.278882+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.278882+0000","objects_scrubbed":2,"log_size":6066,"log_dups_size":0,"ondisk_log_size":6066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:46.218631+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6062,"num_write_kb":49091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9822","reported_seq":10343,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.706720+0000","last_change":"2024-04-25T01:27:47.706744+0000","last_active":"2024-04-25T01:27:47.706720+0000","last_peered":"2024-04-25T01:27:47.706720+0000","last_clean":"2024-04-25T01:27:47.706720+0000","last_became_active":"2024-04-25T01:01:37.441630+0000","last_became_peered":"2024-04-25T01:01:37.441630+0000","last_unstale":"2024-04-25T01:27:47.706720+0000","last_undegraded":"2024-04-25T01:27:47.706720+0000","last_fullsized":"2024-04-25T01:27:47.706720+0000","mapping_epoch":23,"log_start":"27'2453","ondisk_log_start":"27'2453","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9822","last_scrub_stamp":"2024-04-25T01:27:47.706630+0000","last_deep_scrub":"27'9822","last_deep_scrub_stamp":"2024-04-25T01:27:47.706630+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.706630+0000","objects_scrubbed":0,"log_size":7369,"log_dups_size":0,"ondisk_log_size":7369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:32.772238+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":482,"num_read_kb":122428,"num_write":7085,"num_write_kb":850051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2501","reported_seq":2546,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:44.482936+0000","last_change":"2024-04-25T01:27:44.482958+0000","last_active":"2024-04-25T01:27:44.482936+0000","last_peered":"2024-04-25T01:27:44.482936+0000","last_clean":"2024-04-25T01:27:44.482936+0000","last_became_active":"2024-04-25T01:01:33.406331+0000","last_became_peered":"2024-04-25T01:01:33.406331+0000","last_unstale":"2024-04-25T01:27:44.482936+0000","last_undegraded":"2024-04-25T01:27:44.482936+0000","last_fullsized":"2024-04-25T01:27:44.482936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2501","last_scrub_stamp":"2024-04-25T01:27:44.482865+0000","last_deep_scrub":"27'2501","last_deep_scrub_stamp":"2024-04-25T01:27:44.482865+0000","last_clean_scrub_stamp":"2024-04-25T01:27:44.482865+0000","objects_scrubbed":0,"log_size":2501,"log_dups_size":0,"ondisk_log_size":2501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:28.307409+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'7889","reported_seq":7936,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.159998+0000","last_change":"2024-04-25T01:27:35.160056+0000","last_active":"2024-04-25T01:27:35.159998+0000","last_peered":"2024-04-25T01:27:35.159998+0000","last_clean":"2024-04-25T01:27:35.159998+0000","last_became_active":"2024-04-25T01:01:32.393850+0000","last_became_peered":"2024-04-25T01:01:32.393850+0000","last_unstale":"2024-04-25T01:27:35.159998+0000","last_undegraded":"2024-04-25T01:27:35.159998+0000","last_fullsized":"2024-04-25T01:27:35.159998+0000","mapping_epoch":18,"log_start":"27'500","ondisk_log_start":"27'500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7889","last_scrub_stamp":"2024-04-25T01:27:35.159922+0000","last_deep_scrub":"27'7889","last_deep_scrub_stamp":"2024-04-25T01:27:35.159922+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.159922+0000","objects_scrubbed":4,"log_size":7389,"log_dups_size":0,"ondisk_log_size":7389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:10.848084+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7873,"num_write_kb":70555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9288","reported_seq":9787,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.092174+0000","last_change":"2024-04-25T01:27:47.092199+0000","last_active":"2024-04-25T01:27:47.092174+0000","last_peered":"2024-04-25T01:27:47.092174+0000","last_clean":"2024-04-25T01:27:47.092174+0000","last_became_active":"2024-04-25T01:01:37.434774+0000","last_became_peered":"2024-04-25T01:01:37.434774+0000","last_unstale":"2024-04-25T01:27:47.092174+0000","last_undegraded":"2024-04-25T01:27:47.092174+0000","last_fullsized":"2024-04-25T01:27:47.092174+0000","mapping_epoch":23,"log_start":"27'1900","ondisk_log_start":"27'1900","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9288","last_scrub_stamp":"2024-04-25T01:27:47.092096+0000","last_deep_scrub":"27'9288","last_deep_scrub_stamp":"2024-04-25T01:27:47.092096+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.092096+0000","objects_scrubbed":0,"log_size":7388,"log_dups_size":0,"ondisk_log_size":7388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:23.599094+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":460,"num_read_kb":116840,"num_write":6608,"num_write_kb":921401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2540","reported_seq":2585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.697841+0000","last_change":"2024-04-25T01:27:36.697881+0000","last_active":"2024-04-25T01:27:36.697841+0000","last_peered":"2024-04-25T01:27:36.697841+0000","last_clean":"2024-04-25T01:27:36.697841+0000","last_became_active":"2024-04-25T01:01:33.400196+0000","last_became_peered":"2024-04-25T01:01:33.400196+0000","last_unstale":"2024-04-25T01:27:36.697841+0000","last_undegraded":"2024-04-25T01:27:36.697841+0000","last_fullsized":"2024-04-25T01:27:36.697841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2540","last_scrub_stamp":"2024-04-25T01:27:36.697724+0000","last_deep_scrub":"27'2540","last_deep_scrub_stamp":"2024-04-25T01:27:36.697724+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.697724+0000","objects_scrubbed":0,"log_size":2540,"log_dups_size":0,"ondisk_log_size":2540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:57.592417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'8863","reported_seq":8910,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.519468+0000","last_change":"2024-04-25T01:27:37.519514+0000","last_active":"2024-04-25T01:27:37.519468+0000","last_peered":"2024-04-25T01:27:37.519468+0000","last_clean":"2024-04-25T01:27:37.519468+0000","last_became_active":"2024-04-25T01:01:32.397224+0000","last_became_peered":"2024-04-25T01:01:32.397224+0000","last_unstale":"2024-04-25T01:27:37.519468+0000","last_undegraded":"2024-04-25T01:27:37.519468+0000","last_fullsized":"2024-04-25T01:27:37.519468+0000","mapping_epoch":18,"log_start":"27'1500","ondisk_log_start":"27'1500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8863","last_scrub_stamp":"2024-04-25T01:27:37.519370+0000","last_deep_scrub":"27'8863","last_deep_scrub_stamp":"2024-04-25T01:27:37.519370+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.519370+0000","objects_scrubbed":2,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:44.459118+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7408169,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8859,"num_write_kb":62472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9932","reported_seq":10095,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.055164+0000","last_change":"2024-04-25T01:27:46.055205+0000","last_active":"2024-04-25T01:27:46.055164+0000","last_peered":"2024-04-25T01:27:46.055164+0000","last_clean":"2024-04-25T01:27:46.055164+0000","last_became_active":"2024-04-25T01:01:37.432508+0000","last_became_peered":"2024-04-25T01:01:37.432508+0000","last_unstale":"2024-04-25T01:27:46.055164+0000","last_undegraded":"2024-04-25T01:27:46.055164+0000","last_fullsized":"2024-04-25T01:27:46.055164+0000","mapping_epoch":23,"log_start":"27'2523","ondisk_log_start":"27'2523","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9932","last_scrub_stamp":"2024-04-25T01:27:46.055063+0000","last_deep_scrub":"27'9932","last_deep_scrub_stamp":"2024-04-25T01:27:46.055063+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.055063+0000","objects_scrubbed":0,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:05.332722+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":31496,"num_write":7238,"num_write_kb":913890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2545","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.969600+0000","last_change":"2024-04-25T01:27:41.969600+0000","last_active":"2024-04-25T01:27:41.969600+0000","last_peered":"2024-04-25T01:27:41.969600+0000","last_clean":"2024-04-25T01:27:41.969600+0000","last_became_active":"2024-04-25T01:01:33.394121+0000","last_became_peered":"2024-04-25T01:01:33.394121+0000","last_unstale":"2024-04-25T01:27:41.969600+0000","last_undegraded":"2024-04-25T01:27:41.969600+0000","last_fullsized":"2024-04-25T01:27:41.969600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2545","last_scrub_stamp":"2024-04-25T01:27:41.969458+0000","last_deep_scrub":"27'2545","last_deep_scrub_stamp":"2024-04-25T01:27:41.969458+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.969458+0000","objects_scrubbed":0,"log_size":2545,"log_dups_size":0,"ondisk_log_size":2545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:41.861958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7136","reported_seq":8095,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:32.943998+0000","last_change":"2024-04-25T01:27:32.944048+0000","last_active":"2024-04-25T01:27:32.943998+0000","last_peered":"2024-04-25T01:27:32.943998+0000","last_clean":"2024-04-25T01:27:32.943998+0000","last_became_active":"2024-04-25T01:01:32.392340+0000","last_became_peered":"2024-04-25T01:01:32.392340+0000","last_unstale":"2024-04-25T01:27:32.943998+0000","last_undegraded":"2024-04-25T01:27:32.943998+0000","last_fullsized":"2024-04-25T01:27:32.943998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7136","last_scrub_stamp":"2024-04-25T01:27:32.943836+0000","last_deep_scrub":"27'7136","last_deep_scrub_stamp":"2024-04-25T01:27:32.943836+0000","last_clean_scrub_stamp":"2024-04-25T01:27:32.943836+0000","objects_scrubbed":2,"log_size":7136,"log_dups_size":0,"ondisk_log_size":7136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:02.210286+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":912,"num_read_kb":4677,"num_write":7122,"num_write_kb":61966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9066","reported_seq":9501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:48.075347+0000","last_change":"2024-04-25T01:27:48.075390+0000","last_active":"2024-04-25T01:27:48.075347+0000","last_peered":"2024-04-25T01:27:48.075347+0000","last_clean":"2024-04-25T01:27:48.075347+0000","last_became_active":"2024-04-25T01:01:37.435898+0000","last_became_peered":"2024-04-25T01:01:37.435898+0000","last_unstale":"2024-04-25T01:27:48.075347+0000","last_undegraded":"2024-04-25T01:27:48.075347+0000","last_fullsized":"2024-04-25T01:27:48.075347+0000","mapping_epoch":23,"log_start":"27'1700","ondisk_log_start":"27'1700","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9066","last_scrub_stamp":"2024-04-25T01:27:48.075217+0000","last_deep_scrub":"27'9066","last_deep_scrub_stamp":"2024-04-25T01:27:48.075217+0000","last_clean_scrub_stamp":"2024-04-25T01:27:48.075217+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:20.899545+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":100584,"num_write":6328,"num_write_kb":903694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2479","reported_seq":2524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:42.637626+0000","last_change":"2024-04-25T01:27:42.637647+0000","last_active":"2024-04-25T01:27:42.637626+0000","last_peered":"2024-04-25T01:27:42.637626+0000","last_clean":"2024-04-25T01:27:42.637626+0000","last_became_active":"2024-04-25T01:01:33.407284+0000","last_became_peered":"2024-04-25T01:01:33.407284+0000","last_unstale":"2024-04-25T01:27:42.637626+0000","last_undegraded":"2024-04-25T01:27:42.637626+0000","last_fullsized":"2024-04-25T01:27:42.637626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2479","last_scrub_stamp":"2024-04-25T01:27:42.637558+0000","last_deep_scrub":"27'2479","last_deep_scrub_stamp":"2024-04-25T01:27:42.637558+0000","last_clean_scrub_stamp":"2024-04-25T01:27:42.637558+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:22.958730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'6807","reported_seq":6855,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.271097+0000","last_change":"2024-04-25T01:27:40.271097+0000","last_active":"2024-04-25T01:27:40.271097+0000","last_peered":"2024-04-25T01:27:40.271097+0000","last_clean":"2024-04-25T01:27:40.271097+0000","last_became_active":"2024-04-25T01:01:32.399031+0000","last_became_peered":"2024-04-25T01:01:32.399031+0000","last_unstale":"2024-04-25T01:27:40.271097+0000","last_undegraded":"2024-04-25T01:27:40.271097+0000","last_fullsized":"2024-04-25T01:27:40.271097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6807","last_scrub_stamp":"2024-04-25T01:27:40.270952+0000","last_deep_scrub":"27'6807","last_deep_scrub_stamp":"2024-04-25T01:27:40.270952+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.270952+0000","objects_scrubbed":1,"log_size":6807,"log_dups_size":0,"ondisk_log_size":6807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:15.636346+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6806,"num_write_kb":57773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9197","reported_seq":9565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.700480+0000","last_change":"2024-04-25T01:27:45.700525+0000","last_active":"2024-04-25T01:27:45.700480+0000","last_peered":"2024-04-25T01:27:45.700480+0000","last_clean":"2024-04-25T01:27:45.700480+0000","last_became_active":"2024-04-25T01:01:37.440482+0000","last_became_peered":"2024-04-25T01:01:37.440482+0000","last_unstale":"2024-04-25T01:27:45.700480+0000","last_undegraded":"2024-04-25T01:27:45.700480+0000","last_fullsized":"2024-04-25T01:27:45.700480+0000","mapping_epoch":23,"log_start":"27'1826","ondisk_log_start":"27'1826","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9197","last_scrub_stamp":"2024-04-25T01:27:45.700355+0000","last_deep_scrub":"27'9197","last_deep_scrub_stamp":"2024-04-25T01:27:45.700355+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.700355+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:18.524683+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":329,"num_read_kb":83566,"num_write":6535,"num_write_kb":1105233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2598","reported_seq":2643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.316543+0000","last_change":"2024-04-25T01:27:45.316575+0000","last_active":"2024-04-25T01:27:45.316543+0000","last_peered":"2024-04-25T01:27:45.316543+0000","last_clean":"2024-04-25T01:27:45.316543+0000","last_became_active":"2024-04-25T01:01:33.414063+0000","last_became_peered":"2024-04-25T01:01:33.414063+0000","last_unstale":"2024-04-25T01:27:45.316543+0000","last_undegraded":"2024-04-25T01:27:45.316543+0000","last_fullsized":"2024-04-25T01:27:45.316543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2598","last_scrub_stamp":"2024-04-25T01:27:45.316480+0000","last_deep_scrub":"27'2598","last_deep_scrub_stamp":"2024-04-25T01:27:45.316480+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.316480+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:42.718150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4348","reported_seq":4395,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.557974+0000","last_change":"2024-04-25T01:27:38.558000+0000","last_active":"2024-04-25T01:27:38.557974+0000","last_peered":"2024-04-25T01:27:38.557974+0000","last_clean":"2024-04-25T01:27:38.557974+0000","last_became_active":"2024-04-25T01:01:32.397341+0000","last_became_peered":"2024-04-25T01:01:32.397341+0000","last_unstale":"2024-04-25T01:27:38.557974+0000","last_undegraded":"2024-04-25T01:27:38.557974+0000","last_fullsized":"2024-04-25T01:27:38.557974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4348","last_scrub_stamp":"2024-04-25T01:27:38.557894+0000","last_deep_scrub":"27'4348","last_deep_scrub_stamp":"2024-04-25T01:27:38.557894+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.557894+0000","objects_scrubbed":1,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:13.923288+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4347,"num_write_kb":39756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":206,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.353649+0000","last_change":"2024-04-25T01:27:36.353674+0000","last_active":"2024-04-25T01:27:36.353649+0000","last_peered":"2024-04-25T01:27:36.353649+0000","last_clean":"2024-04-25T01:27:36.353649+0000","last_became_active":"2024-04-25T01:01:22.651883+0000","last_became_peered":"2024-04-25T01:01:22.651883+0000","last_unstale":"2024-04-25T01:27:36.353649+0000","last_undegraded":"2024-04-25T01:27:36.353649+0000","last_fullsized":"2024-04-25T01:27:36.353649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-25T01:27:36.353589+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-25T01:27:36.353589+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.353589+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:44.006736+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'9541","reported_seq":9811,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:44.948209+0000","last_change":"2024-04-25T01:27:44.948235+0000","last_active":"2024-04-25T01:27:44.948209+0000","last_peered":"2024-04-25T01:27:44.948209+0000","last_clean":"2024-04-25T01:27:44.948209+0000","last_became_active":"2024-04-25T01:01:37.430621+0000","last_became_peered":"2024-04-25T01:01:37.430621+0000","last_unstale":"2024-04-25T01:27:44.948209+0000","last_undegraded":"2024-04-25T01:27:44.948209+0000","last_fullsized":"2024-04-25T01:27:44.948209+0000","mapping_epoch":23,"log_start":"27'1566","ondisk_log_start":"27'1566","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9541","last_scrub_stamp":"2024-04-25T01:27:44.948122+0000","last_deep_scrub":"27'9541","last_deep_scrub_stamp":"2024-04-25T01:27:44.948122+0000","last_clean_scrub_stamp":"2024-04-25T01:27:44.948122+0000","objects_scrubbed":0,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:03.026479+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":58674,"num_write":6904,"num_write_kb":915015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2524","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:50.304545+0000","last_change":"2024-04-25T01:27:50.304583+0000","last_active":"2024-04-25T01:27:50.304545+0000","last_peered":"2024-04-25T01:27:50.304545+0000","last_clean":"2024-04-25T01:27:50.304545+0000","last_became_active":"2024-04-25T01:01:33.409682+0000","last_became_peered":"2024-04-25T01:01:33.409682+0000","last_unstale":"2024-04-25T01:27:50.304545+0000","last_undegraded":"2024-04-25T01:27:50.304545+0000","last_fullsized":"2024-04-25T01:27:50.304545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2524","last_scrub_stamp":"2024-04-25T01:27:50.304445+0000","last_deep_scrub":"27'2524","last_deep_scrub_stamp":"2024-04-25T01:27:50.304445+0000","last_clean_scrub_stamp":"2024-04-25T01:27:50.304445+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:33.661645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7386","reported_seq":7432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.997865+0000","last_change":"2024-04-25T01:27:35.997898+0000","last_active":"2024-04-25T01:27:35.997865+0000","last_peered":"2024-04-25T01:27:35.997865+0000","last_clean":"2024-04-25T01:27:35.997865+0000","last_became_active":"2024-04-25T01:01:32.399259+0000","last_became_peered":"2024-04-25T01:01:32.399259+0000","last_unstale":"2024-04-25T01:27:35.997865+0000","last_undegraded":"2024-04-25T01:27:35.997865+0000","last_fullsized":"2024-04-25T01:27:35.997865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7386","last_scrub_stamp":"2024-04-25T01:27:35.997796+0000","last_deep_scrub":"27'7386","last_deep_scrub_stamp":"2024-04-25T01:27:35.997796+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.997796+0000","objects_scrubbed":2,"log_size":7386,"log_dups_size":0,"ondisk_log_size":7386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:42:54.183673+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7389,"num_write_kb":58188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9283","reported_seq":9797,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:48.854837+0000","last_change":"2024-04-25T01:27:48.854837+0000","last_active":"2024-04-25T01:27:48.854837+0000","last_peered":"2024-04-25T01:27:48.854837+0000","last_clean":"2024-04-25T01:27:48.854837+0000","last_became_active":"2024-04-25T01:02:13.829268+0000","last_became_peered":"2024-04-25T01:02:13.829268+0000","last_unstale":"2024-04-25T01:27:48.854837+0000","last_undegraded":"2024-04-25T01:27:48.854837+0000","last_fullsized":"2024-04-25T01:27:48.854837+0000","mapping_epoch":26,"log_start":"27'1312","ondisk_log_start":"27'1312","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9283","last_scrub_stamp":"2024-04-25T01:27:48.854727+0000","last_deep_scrub":"27'9283","last_deep_scrub_stamp":"2024-04-25T01:27:48.854727+0000","last_clean_scrub_stamp":"2024-04-25T01:27:48.854727+0000","objects_scrubbed":0,"log_size":7971,"log_dups_size":0,"ondisk_log_size":7971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:25.361725+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":468,"num_read_kb":118872,"num_write":6604,"num_write_kb":925635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2554","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.496255+0000","last_change":"2024-04-25T01:27:45.496278+0000","last_active":"2024-04-25T01:27:45.496255+0000","last_peered":"2024-04-25T01:27:45.496255+0000","last_clean":"2024-04-25T01:27:45.496255+0000","last_became_active":"2024-04-25T01:01:33.411544+0000","last_became_peered":"2024-04-25T01:01:33.411544+0000","last_unstale":"2024-04-25T01:27:45.496255+0000","last_undegraded":"2024-04-25T01:27:45.496255+0000","last_fullsized":"2024-04-25T01:27:45.496255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2554","last_scrub_stamp":"2024-04-25T01:27:45.496187+0000","last_deep_scrub":"27'2554","last_deep_scrub_stamp":"2024-04-25T01:27:45.496187+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.496187+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:55.102021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7036","reported_seq":7082,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.082099+0000","last_change":"2024-04-25T01:27:34.082128+0000","last_active":"2024-04-25T01:27:34.082099+0000","last_peered":"2024-04-25T01:27:34.082099+0000","last_clean":"2024-04-25T01:27:34.082099+0000","last_became_active":"2024-04-25T01:01:32.391327+0000","last_became_peered":"2024-04-25T01:01:32.391327+0000","last_unstale":"2024-04-25T01:27:34.082099+0000","last_undegraded":"2024-04-25T01:27:34.082099+0000","last_fullsized":"2024-04-25T01:27:34.082099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7036","last_scrub_stamp":"2024-04-25T01:27:34.082033+0000","last_deep_scrub":"27'7036","last_deep_scrub_stamp":"2024-04-25T01:27:34.082033+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.082033+0000","objects_scrubbed":3,"log_size":7036,"log_dups_size":0,"ondisk_log_size":7036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:52.634087+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7035,"num_write_kb":58075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9471","reported_seq":10173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.734729+0000","last_change":"2024-04-25T01:27:46.734753+0000","last_active":"2024-04-25T01:27:46.734729+0000","last_peered":"2024-04-25T01:27:46.734729+0000","last_clean":"2024-04-25T01:27:46.734729+0000","last_became_active":"2024-04-25T01:01:37.440559+0000","last_became_peered":"2024-04-25T01:01:37.440559+0000","last_unstale":"2024-04-25T01:27:46.734729+0000","last_undegraded":"2024-04-25T01:27:46.734729+0000","last_fullsized":"2024-04-25T01:27:46.734729+0000","mapping_epoch":23,"log_start":"27'2138","ondisk_log_start":"27'2138","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9471","last_scrub_stamp":"2024-04-25T01:27:46.734642+0000","last_deep_scrub":"27'9471","last_deep_scrub_stamp":"2024-04-25T01:27:46.734642+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.734642+0000","objects_scrubbed":0,"log_size":7333,"log_dups_size":0,"ondisk_log_size":7333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:00.252496+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":664,"num_read_kb":168656,"num_write":6719,"num_write_kb":895554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2424","reported_seq":2468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.396275+0000","last_change":"2024-04-25T01:27:35.396299+0000","last_active":"2024-04-25T01:27:35.396275+0000","last_peered":"2024-04-25T01:27:35.396275+0000","last_clean":"2024-04-25T01:27:35.396275+0000","last_became_active":"2024-04-25T01:01:33.389766+0000","last_became_peered":"2024-04-25T01:01:33.389766+0000","last_unstale":"2024-04-25T01:27:35.396275+0000","last_undegraded":"2024-04-25T01:27:35.396275+0000","last_fullsized":"2024-04-25T01:27:35.396275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2424","last_scrub_stamp":"2024-04-25T01:27:35.396190+0000","last_deep_scrub":"27'2424","last_deep_scrub_stamp":"2024-04-25T01:27:35.396190+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.396190+0000","objects_scrubbed":0,"log_size":2424,"log_dups_size":0,"ondisk_log_size":2424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:58.614683+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8748","reported_seq":8794,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.032656+0000","last_change":"2024-04-25T01:27:34.032724+0000","last_active":"2024-04-25T01:27:34.032656+0000","last_peered":"2024-04-25T01:27:34.032656+0000","last_clean":"2024-04-25T01:27:34.032656+0000","last_became_active":"2024-04-25T01:01:32.392869+0000","last_became_peered":"2024-04-25T01:01:32.392869+0000","last_unstale":"2024-04-25T01:27:34.032656+0000","last_undegraded":"2024-04-25T01:27:34.032656+0000","last_fullsized":"2024-04-25T01:27:34.032656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8748","last_scrub_stamp":"2024-04-25T01:27:34.032584+0000","last_deep_scrub":"27'8748","last_deep_scrub_stamp":"2024-04-25T01:27:34.032584+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.032584+0000","objects_scrubbed":7,"log_size":8748,"log_dups_size":0,"ondisk_log_size":8748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:32.310929+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8749,"num_write_kb":79319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9302","reported_seq":10376,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:51.528746+0000","last_change":"2024-04-25T01:27:51.528793+0000","last_active":"2024-04-25T01:27:51.528746+0000","last_peered":"2024-04-25T01:27:51.528746+0000","last_clean":"2024-04-25T01:27:51.528746+0000","last_became_active":"2024-04-25T01:01:37.431009+0000","last_became_peered":"2024-04-25T01:01:37.431009+0000","last_unstale":"2024-04-25T01:27:51.528746+0000","last_undegraded":"2024-04-25T01:27:51.528746+0000","last_fullsized":"2024-04-25T01:27:51.528746+0000","mapping_epoch":23,"log_start":"27'1886","ondisk_log_start":"27'1886","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9302","last_scrub_stamp":"2024-04-25T01:27:51.528621+0000","last_deep_scrub":"27'9302","last_deep_scrub_stamp":"2024-04-25T01:27:51.528621+0000","last_clean_scrub_stamp":"2024-04-25T01:27:51.528621+0000","objects_scrubbed":0,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:46.293745+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1035,"num_read_kb":262890,"num_write":6564,"num_write_kb":1048321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2520","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:44.334478+0000","last_change":"2024-04-25T01:27:44.334498+0000","last_active":"2024-04-25T01:27:44.334478+0000","last_peered":"2024-04-25T01:27:44.334478+0000","last_clean":"2024-04-25T01:27:44.334478+0000","last_became_active":"2024-04-25T01:01:33.389384+0000","last_became_peered":"2024-04-25T01:01:33.389384+0000","last_unstale":"2024-04-25T01:27:44.334478+0000","last_undegraded":"2024-04-25T01:27:44.334478+0000","last_fullsized":"2024-04-25T01:27:44.334478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2520","last_scrub_stamp":"2024-04-25T01:27:44.334408+0000","last_deep_scrub":"27'2520","last_deep_scrub_stamp":"2024-04-25T01:27:44.334408+0000","last_clean_scrub_stamp":"2024-04-25T01:27:44.334408+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:02.412765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5710","reported_seq":5756,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:33.701012+0000","last_change":"2024-04-25T01:27:33.701039+0000","last_active":"2024-04-25T01:27:33.701012+0000","last_peered":"2024-04-25T01:27:33.701012+0000","last_clean":"2024-04-25T01:27:33.701012+0000","last_became_active":"2024-04-25T01:01:32.394154+0000","last_became_peered":"2024-04-25T01:01:32.394154+0000","last_unstale":"2024-04-25T01:27:33.701012+0000","last_undegraded":"2024-04-25T01:27:33.701012+0000","last_fullsized":"2024-04-25T01:27:33.701012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5710","last_scrub_stamp":"2024-04-25T01:27:33.700911+0000","last_deep_scrub":"27'5710","last_deep_scrub_stamp":"2024-04-25T01:27:33.700911+0000","last_clean_scrub_stamp":"2024-04-25T01:27:33.700911+0000","objects_scrubbed":2,"log_size":5710,"log_dups_size":0,"ondisk_log_size":5710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:33:21.312854+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":44693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'8871","reported_seq":9304,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.681628+0000","last_change":"2024-04-25T01:27:46.681628+0000","last_active":"2024-04-25T01:27:46.681628+0000","last_peered":"2024-04-25T01:27:46.681628+0000","last_clean":"2024-04-25T01:27:46.681628+0000","last_became_active":"2024-04-25T01:01:37.439908+0000","last_became_peered":"2024-04-25T01:01:37.439908+0000","last_unstale":"2024-04-25T01:27:46.681628+0000","last_undegraded":"2024-04-25T01:27:46.681628+0000","last_fullsized":"2024-04-25T01:27:46.681628+0000","mapping_epoch":23,"log_start":"27'1713","ondisk_log_start":"27'1713","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8871","last_scrub_stamp":"2024-04-25T01:27:46.681490+0000","last_deep_scrub":"27'8871","last_deep_scrub_stamp":"2024-04-25T01:27:46.681490+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.681490+0000","objects_scrubbed":0,"log_size":7158,"log_dups_size":0,"ondisk_log_size":7158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:27.002662+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":394,"num_read_kb":100076,"num_write":6161,"num_write_kb":854384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2599","reported_seq":2643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.295421+0000","last_change":"2024-04-25T01:27:43.295444+0000","last_active":"2024-04-25T01:27:43.295421+0000","last_peered":"2024-04-25T01:27:43.295421+0000","last_clean":"2024-04-25T01:27:43.295421+0000","last_became_active":"2024-04-25T01:01:33.388433+0000","last_became_peered":"2024-04-25T01:01:33.388433+0000","last_unstale":"2024-04-25T01:27:43.295421+0000","last_undegraded":"2024-04-25T01:27:43.295421+0000","last_fullsized":"2024-04-25T01:27:43.295421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2599","last_scrub_stamp":"2024-04-25T01:27:43.295354+0000","last_deep_scrub":"27'2599","last_deep_scrub_stamp":"2024-04-25T01:27:43.295354+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.295354+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:47.248220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'5138","reported_seq":5184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.012984+0000","last_change":"2024-04-25T01:27:35.013024+0000","last_active":"2024-04-25T01:27:35.012984+0000","last_peered":"2024-04-25T01:27:35.012984+0000","last_clean":"2024-04-25T01:27:35.012984+0000","last_became_active":"2024-04-25T01:01:32.393859+0000","last_became_peered":"2024-04-25T01:01:32.393859+0000","last_unstale":"2024-04-25T01:27:35.012984+0000","last_undegraded":"2024-04-25T01:27:35.012984+0000","last_fullsized":"2024-04-25T01:27:35.012984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5138","last_scrub_stamp":"2024-04-25T01:27:35.012911+0000","last_deep_scrub":"27'5138","last_deep_scrub_stamp":"2024-04-25T01:27:35.012911+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.012911+0000","objects_scrubbed":6,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:46:36.786079+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5139,"num_write_kb":40185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9389","reported_seq":9734,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:44.678798+0000","last_change":"2024-04-25T01:27:44.678823+0000","last_active":"2024-04-25T01:27:44.678798+0000","last_peered":"2024-04-25T01:27:44.678798+0000","last_clean":"2024-04-25T01:27:44.678798+0000","last_became_active":"2024-04-25T01:01:37.436046+0000","last_became_peered":"2024-04-25T01:01:37.436046+0000","last_unstale":"2024-04-25T01:27:44.678798+0000","last_undegraded":"2024-04-25T01:27:44.678798+0000","last_fullsized":"2024-04-25T01:27:44.678798+0000","mapping_epoch":23,"log_start":"27'2023","ondisk_log_start":"27'2023","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9389","last_scrub_stamp":"2024-04-25T01:27:44.678709+0000","last_deep_scrub":"27'9389","last_deep_scrub_stamp":"2024-04-25T01:27:44.678709+0000","last_clean_scrub_stamp":"2024-04-25T01:27:44.678709+0000","objects_scrubbed":0,"log_size":7366,"log_dups_size":0,"ondisk_log_size":7366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:21.185706+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":78232,"num_write":6705,"num_write_kb":1182057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2468","reported_seq":2512,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.946536+0000","last_change":"2024-04-25T01:27:40.946572+0000","last_active":"2024-04-25T01:27:40.946536+0000","last_peered":"2024-04-25T01:27:40.946536+0000","last_clean":"2024-04-25T01:27:40.946536+0000","last_became_active":"2024-04-25T01:01:33.402108+0000","last_became_peered":"2024-04-25T01:01:33.402108+0000","last_unstale":"2024-04-25T01:27:40.946536+0000","last_undegraded":"2024-04-25T01:27:40.946536+0000","last_fullsized":"2024-04-25T01:27:40.946536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2468","last_scrub_stamp":"2024-04-25T01:27:40.946460+0000","last_deep_scrub":"27'2468","last_deep_scrub_stamp":"2024-04-25T01:27:40.946460+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.946460+0000","objects_scrubbed":0,"log_size":2468,"log_dups_size":0,"ondisk_log_size":2468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:13.804346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4847","reported_seq":4893,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.007207+0000","last_change":"2024-04-25T01:27:37.007240+0000","last_active":"2024-04-25T01:27:37.007207+0000","last_peered":"2024-04-25T01:27:37.007207+0000","last_clean":"2024-04-25T01:27:37.007207+0000","last_became_active":"2024-04-25T01:01:32.397634+0000","last_became_peered":"2024-04-25T01:01:32.397634+0000","last_unstale":"2024-04-25T01:27:37.007207+0000","last_undegraded":"2024-04-25T01:27:37.007207+0000","last_fullsized":"2024-04-25T01:27:37.007207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4847","last_scrub_stamp":"2024-04-25T01:27:37.007147+0000","last_deep_scrub":"27'4847","last_deep_scrub_stamp":"2024-04-25T01:27:37.007147+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.007147+0000","objects_scrubbed":3,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:00.225640+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4848,"num_write_kb":44079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9127","reported_seq":9637,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:51.443000+0000","last_change":"2024-04-25T01:27:51.443023+0000","last_active":"2024-04-25T01:27:51.443000+0000","last_peered":"2024-04-25T01:27:51.443000+0000","last_clean":"2024-04-25T01:27:51.443000+0000","last_became_active":"2024-04-25T01:01:37.441672+0000","last_became_peered":"2024-04-25T01:01:37.441672+0000","last_unstale":"2024-04-25T01:27:51.443000+0000","last_undegraded":"2024-04-25T01:27:51.443000+0000","last_fullsized":"2024-04-25T01:27:51.443000+0000","mapping_epoch":23,"log_start":"27'2828","ondisk_log_start":"27'2828","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9127","last_scrub_stamp":"2024-04-25T01:27:51.442917+0000","last_deep_scrub":"27'9127","last_deep_scrub_stamp":"2024-04-25T01:27:51.442917+0000","last_clean_scrub_stamp":"2024-04-25T01:27:51.442917+0000","objects_scrubbed":0,"log_size":6299,"log_dups_size":0,"ondisk_log_size":6299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:00:48.387719+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":473,"num_read_kb":120142,"num_write":6396,"num_write_kb":909662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2571","reported_seq":2600,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.991745+0000","last_change":"2024-04-25T01:27:39.991768+0000","last_active":"2024-04-25T01:27:39.991745+0000","last_peered":"2024-04-25T01:27:39.991745+0000","last_clean":"2024-04-25T01:27:39.991745+0000","last_became_active":"2024-04-25T01:02:13.743832+0000","last_became_peered":"2024-04-25T01:02:13.743832+0000","last_unstale":"2024-04-25T01:27:39.991745+0000","last_undegraded":"2024-04-25T01:27:39.991745+0000","last_fullsized":"2024-04-25T01:27:39.991745+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2571","last_scrub_stamp":"2024-04-25T01:27:39.991660+0000","last_deep_scrub":"27'2571","last_deep_scrub_stamp":"2024-04-25T01:27:39.991660+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.991660+0000","objects_scrubbed":0,"log_size":2571,"log_dups_size":0,"ondisk_log_size":2571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:32.674114+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"27'2888","reported_seq":2934,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.609059+0000","last_change":"2024-04-25T01:27:39.609104+0000","last_active":"2024-04-25T01:27:39.609059+0000","last_peered":"2024-04-25T01:27:39.609059+0000","last_clean":"2024-04-25T01:27:39.609059+0000","last_became_active":"2024-04-25T01:01:32.397782+0000","last_became_peered":"2024-04-25T01:01:32.397782+0000","last_unstale":"2024-04-25T01:27:39.609059+0000","last_undegraded":"2024-04-25T01:27:39.609059+0000","last_fullsized":"2024-04-25T01:27:39.609059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2888","last_scrub_stamp":"2024-04-25T01:27:39.608983+0000","last_deep_scrub":"27'2888","last_deep_scrub_stamp":"2024-04-25T01:27:39.608983+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.608983+0000","objects_scrubbed":2,"log_size":2888,"log_dups_size":0,"ondisk_log_size":2888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:44.240826+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2884,"num_write_kb":30449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9238","reported_seq":9490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:52.474747+0000","last_change":"2024-04-25T01:27:52.474770+0000","last_active":"2024-04-25T01:27:52.474747+0000","last_peered":"2024-04-25T01:27:52.474747+0000","last_clean":"2024-04-25T01:27:52.474747+0000","last_became_active":"2024-04-25T01:02:13.829847+0000","last_became_peered":"2024-04-25T01:02:13.829847+0000","last_unstale":"2024-04-25T01:27:52.474747+0000","last_undegraded":"2024-04-25T01:27:52.474747+0000","last_fullsized":"2024-04-25T01:27:52.474747+0000","mapping_epoch":26,"log_start":"27'2967","ondisk_log_start":"27'2967","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9238","last_scrub_stamp":"2024-04-25T01:27:52.474656+0000","last_deep_scrub":"27'9238","last_deep_scrub_stamp":"2024-04-25T01:27:52.474656+0000","last_clean_scrub_stamp":"2024-04-25T01:27:52.474656+0000","objects_scrubbed":0,"log_size":6271,"log_dups_size":0,"ondisk_log_size":6271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:19.745543+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":207,"num_read_kb":52578,"num_write":6628,"num_write_kb":923125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2535","reported_seq":2579,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.685889+0000","last_change":"2024-04-25T01:27:40.685912+0000","last_active":"2024-04-25T01:27:40.685889+0000","last_peered":"2024-04-25T01:27:40.685889+0000","last_clean":"2024-04-25T01:27:40.685889+0000","last_became_active":"2024-04-25T01:01:33.404012+0000","last_became_peered":"2024-04-25T01:01:33.404012+0000","last_unstale":"2024-04-25T01:27:40.685889+0000","last_undegraded":"2024-04-25T01:27:40.685889+0000","last_fullsized":"2024-04-25T01:27:40.685889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2535","last_scrub_stamp":"2024-04-25T01:27:40.685818+0000","last_deep_scrub":"27'2535","last_deep_scrub_stamp":"2024-04-25T01:27:40.685818+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.685818+0000","objects_scrubbed":0,"log_size":2535,"log_dups_size":0,"ondisk_log_size":2535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:58.764175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6779","reported_seq":6825,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:33.694066+0000","last_change":"2024-04-25T01:27:33.694129+0000","last_active":"2024-04-25T01:27:33.694066+0000","last_peered":"2024-04-25T01:27:33.694066+0000","last_clean":"2024-04-25T01:27:33.694066+0000","last_became_active":"2024-04-25T01:01:32.391163+0000","last_became_peered":"2024-04-25T01:01:32.391163+0000","last_unstale":"2024-04-25T01:27:33.694066+0000","last_undegraded":"2024-04-25T01:27:33.694066+0000","last_fullsized":"2024-04-25T01:27:33.694066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6779","last_scrub_stamp":"2024-04-25T01:27:33.693993+0000","last_deep_scrub":"27'6779","last_deep_scrub_stamp":"2024-04-25T01:27:33.693993+0000","last_clean_scrub_stamp":"2024-04-25T01:27:33.693993+0000","objects_scrubbed":7,"log_size":6779,"log_dups_size":0,"ondisk_log_size":6779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:09.382155+0000","scrub_duration":66,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6781,"num_write_kb":57696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9620","reported_seq":10006,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:52.480241+0000","last_change":"2024-04-25T01:27:52.480265+0000","last_active":"2024-04-25T01:27:52.480241+0000","last_peered":"2024-04-25T01:27:52.480241+0000","last_clean":"2024-04-25T01:27:52.480241+0000","last_became_active":"2024-04-25T01:01:37.437856+0000","last_became_peered":"2024-04-25T01:01:37.437856+0000","last_unstale":"2024-04-25T01:27:52.480241+0000","last_undegraded":"2024-04-25T01:27:52.480241+0000","last_fullsized":"2024-04-25T01:27:52.480241+0000","mapping_epoch":23,"log_start":"27'2214","ondisk_log_start":"27'2214","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9620","last_scrub_stamp":"2024-04-25T01:27:52.480171+0000","last_deep_scrub":"27'9620","last_deep_scrub_stamp":"2024-04-25T01:27:52.480171+0000","last_clean_scrub_stamp":"2024-04-25T01:27:52.480171+0000","objects_scrubbed":0,"log_size":7406,"log_dups_size":0,"ondisk_log_size":7406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:19.858557+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":349,"num_read_kb":88646,"num_write":6988,"num_write_kb":1079633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2474","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.288818+0000","last_change":"2024-04-25T01:27:47.288860+0000","last_active":"2024-04-25T01:27:47.288818+0000","last_peered":"2024-04-25T01:27:47.288818+0000","last_clean":"2024-04-25T01:27:47.288818+0000","last_became_active":"2024-04-25T01:01:33.395703+0000","last_became_peered":"2024-04-25T01:01:33.395703+0000","last_unstale":"2024-04-25T01:27:47.288818+0000","last_undegraded":"2024-04-25T01:27:47.288818+0000","last_fullsized":"2024-04-25T01:27:47.288818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2474","last_scrub_stamp":"2024-04-25T01:27:47.288720+0000","last_deep_scrub":"27'2474","last_deep_scrub_stamp":"2024-04-25T01:27:47.288720+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.288720+0000","objects_scrubbed":0,"log_size":2474,"log_dups_size":0,"ondisk_log_size":2474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:56.894555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10288","reported_seq":10927,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.024027+0000","last_change":"2024-04-25T01:27:35.024027+0000","last_active":"2024-04-25T01:27:35.024027+0000","last_peered":"2024-04-25T01:27:35.024027+0000","last_clean":"2024-04-25T01:27:35.024027+0000","last_became_active":"2024-04-25T01:01:32.394130+0000","last_became_peered":"2024-04-25T01:01:32.394130+0000","last_unstale":"2024-04-25T01:27:35.024027+0000","last_undegraded":"2024-04-25T01:27:35.024027+0000","last_fullsized":"2024-04-25T01:27:35.024027+0000","mapping_epoch":18,"log_start":"27'2300","ondisk_log_start":"27'2300","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10287","last_scrub_stamp":"2024-04-25T01:27:35.023923+0000","last_deep_scrub":"27'10287","last_deep_scrub_stamp":"2024-04-25T01:27:35.023923+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.023923+0000","objects_scrubbed":5,"log_size":7988,"log_dups_size":0,"ondisk_log_size":7988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:12.908928+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":594,"num_read_kb":4482,"num_write":10278,"num_write_kb":76427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9154","reported_seq":9504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.058268+0000","last_change":"2024-04-25T01:27:45.058324+0000","last_active":"2024-04-25T01:27:45.058268+0000","last_peered":"2024-04-25T01:27:45.058268+0000","last_clean":"2024-04-25T01:27:45.058268+0000","last_became_active":"2024-04-25T01:01:37.431248+0000","last_became_peered":"2024-04-25T01:01:37.431248+0000","last_unstale":"2024-04-25T01:27:45.058268+0000","last_undegraded":"2024-04-25T01:27:45.058268+0000","last_fullsized":"2024-04-25T01:27:45.058268+0000","mapping_epoch":23,"log_start":"27'1834","ondisk_log_start":"27'1834","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9154","last_scrub_stamp":"2024-04-25T01:27:45.058083+0000","last_deep_scrub":"27'9154","last_deep_scrub_stamp":"2024-04-25T01:27:45.058083+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.058083+0000","objects_scrubbed":0,"log_size":7320,"log_dups_size":0,"ondisk_log_size":7320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:25.379255+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":312,"num_read_kb":79248,"num_write":6491,"num_write_kb":938399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'2554","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.668998+0000","last_change":"2024-04-25T01:27:35.669036+0000","last_active":"2024-04-25T01:27:35.668998+0000","last_peered":"2024-04-25T01:27:35.668998+0000","last_clean":"2024-04-25T01:27:35.668998+0000","last_became_active":"2024-04-25T01:01:33.388077+0000","last_became_peered":"2024-04-25T01:01:33.388077+0000","last_unstale":"2024-04-25T01:27:35.668998+0000","last_undegraded":"2024-04-25T01:27:35.668998+0000","last_fullsized":"2024-04-25T01:27:35.668998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2554","last_scrub_stamp":"2024-04-25T01:27:35.668892+0000","last_deep_scrub":"27'2554","last_deep_scrub_stamp":"2024-04-25T01:27:35.668892+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.668892+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:54.220946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6458","reported_seq":6503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.005831+0000","last_change":"2024-04-25T01:27:38.005855+0000","last_active":"2024-04-25T01:27:38.005831+0000","last_peered":"2024-04-25T01:27:38.005831+0000","last_clean":"2024-04-25T01:27:38.005831+0000","last_became_active":"2024-04-25T01:01:32.397566+0000","last_became_peered":"2024-04-25T01:01:32.397566+0000","last_unstale":"2024-04-25T01:27:38.005831+0000","last_undegraded":"2024-04-25T01:27:38.005831+0000","last_fullsized":"2024-04-25T01:27:38.005831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6458","last_scrub_stamp":"2024-04-25T01:27:38.005772+0000","last_deep_scrub":"27'6458","last_deep_scrub_stamp":"2024-04-25T01:27:38.005772+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.005772+0000","objects_scrubbed":2,"log_size":6458,"log_dups_size":0,"ondisk_log_size":6458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:44.945520+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6472,"num_write_kb":41579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'8967","reported_seq":9513,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.650659+0000","last_change":"2024-04-25T01:27:47.650683+0000","last_active":"2024-04-25T01:27:47.650659+0000","last_peered":"2024-04-25T01:27:47.650659+0000","last_clean":"2024-04-25T01:27:47.650659+0000","last_became_active":"2024-04-25T01:01:37.440469+0000","last_became_peered":"2024-04-25T01:01:37.440469+0000","last_unstale":"2024-04-25T01:27:47.650659+0000","last_undegraded":"2024-04-25T01:27:47.650659+0000","last_fullsized":"2024-04-25T01:27:47.650659+0000","mapping_epoch":23,"log_start":"27'1803","ondisk_log_start":"27'1803","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8967","last_scrub_stamp":"2024-04-25T01:27:47.650568+0000","last_deep_scrub":"27'8967","last_deep_scrub_stamp":"2024-04-25T01:27:47.650568+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.650568+0000","objects_scrubbed":0,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:06.928226+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":508,"num_read_kb":129032,"num_write":6269,"num_write_kb":1000518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2578","reported_seq":2621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.694518+0000","last_change":"2024-04-25T01:27:39.694561+0000","last_active":"2024-04-25T01:27:39.694518+0000","last_peered":"2024-04-25T01:27:39.694518+0000","last_clean":"2024-04-25T01:27:39.694518+0000","last_became_active":"2024-04-25T01:01:33.403394+0000","last_became_peered":"2024-04-25T01:01:33.403394+0000","last_unstale":"2024-04-25T01:27:39.694518+0000","last_undegraded":"2024-04-25T01:27:39.694518+0000","last_fullsized":"2024-04-25T01:27:39.694518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2578","last_scrub_stamp":"2024-04-25T01:27:39.694410+0000","last_deep_scrub":"27'2578","last_deep_scrub_stamp":"2024-04-25T01:27:39.694410+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.694410+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:39.586910+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6615","reported_seq":6660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.572285+0000","last_change":"2024-04-25T01:27:40.572334+0000","last_active":"2024-04-25T01:27:40.572285+0000","last_peered":"2024-04-25T01:27:40.572285+0000","last_clean":"2024-04-25T01:27:40.572285+0000","last_became_active":"2024-04-25T01:01:32.397773+0000","last_became_peered":"2024-04-25T01:01:32.397773+0000","last_unstale":"2024-04-25T01:27:40.572285+0000","last_undegraded":"2024-04-25T01:27:40.572285+0000","last_fullsized":"2024-04-25T01:27:40.572285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6615","last_scrub_stamp":"2024-04-25T01:27:40.572181+0000","last_deep_scrub":"27'6615","last_deep_scrub_stamp":"2024-04-25T01:27:40.572181+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.572181+0000","objects_scrubbed":2,"log_size":6615,"log_dups_size":0,"ondisk_log_size":6615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:32.097667+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6608,"num_write_kb":53556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9229","reported_seq":9445,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.382985+0000","last_change":"2024-04-25T01:27:46.383029+0000","last_active":"2024-04-25T01:27:46.382985+0000","last_peered":"2024-04-25T01:27:46.382985+0000","last_clean":"2024-04-25T01:27:46.382985+0000","last_became_active":"2024-04-25T01:01:37.432623+0000","last_became_peered":"2024-04-25T01:01:37.432623+0000","last_unstale":"2024-04-25T01:27:46.382985+0000","last_undegraded":"2024-04-25T01:27:46.382985+0000","last_fullsized":"2024-04-25T01:27:46.382985+0000","mapping_epoch":23,"log_start":"27'2910","ondisk_log_start":"27'2910","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9229","last_scrub_stamp":"2024-04-25T01:27:46.382832+0000","last_deep_scrub":"27'9229","last_deep_scrub_stamp":"2024-04-25T01:27:46.382832+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.382832+0000","objects_scrubbed":0,"log_size":6319,"log_dups_size":0,"ondisk_log_size":6319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.860307+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":45212,"num_write":6556,"num_write_kb":963248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2548","reported_seq":2591,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:49.443332+0000","last_change":"2024-04-25T01:27:49.443354+0000","last_active":"2024-04-25T01:27:49.443332+0000","last_peered":"2024-04-25T01:27:49.443332+0000","last_clean":"2024-04-25T01:27:49.443332+0000","last_became_active":"2024-04-25T01:01:33.412989+0000","last_became_peered":"2024-04-25T01:01:33.412989+0000","last_unstale":"2024-04-25T01:27:49.443332+0000","last_undegraded":"2024-04-25T01:27:49.443332+0000","last_fullsized":"2024-04-25T01:27:49.443332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2548","last_scrub_stamp":"2024-04-25T01:27:49.443260+0000","last_deep_scrub":"27'2548","last_deep_scrub_stamp":"2024-04-25T01:27:49.443260+0000","last_clean_scrub_stamp":"2024-04-25T01:27:49.443260+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:19.563962+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'6623","reported_seq":7487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.990229+0000","last_change":"2024-04-25T01:27:35.990251+0000","last_active":"2024-04-25T01:27:35.990229+0000","last_peered":"2024-04-25T01:27:35.990229+0000","last_clean":"2024-04-25T01:27:35.990229+0000","last_became_active":"2024-04-25T01:01:32.398088+0000","last_became_peered":"2024-04-25T01:01:32.398088+0000","last_unstale":"2024-04-25T01:27:35.990229+0000","last_undegraded":"2024-04-25T01:27:35.990229+0000","last_fullsized":"2024-04-25T01:27:35.990229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6623","last_scrub_stamp":"2024-04-25T01:27:35.990160+0000","last_deep_scrub":"27'6623","last_deep_scrub_stamp":"2024-04-25T01:27:35.990160+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.990160+0000","objects_scrubbed":0,"log_size":6623,"log_dups_size":0,"ondisk_log_size":6623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:28.672614+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":821,"num_read_kb":4618,"num_write":6625,"num_write_kb":49469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'10011","reported_seq":10084,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:49.439797+0000","last_change":"2024-04-25T01:27:49.439843+0000","last_active":"2024-04-25T01:27:49.439797+0000","last_peered":"2024-04-25T01:27:49.439797+0000","last_clean":"2024-04-25T01:27:49.439797+0000","last_became_active":"2024-04-25T01:01:37.440218+0000","last_became_peered":"2024-04-25T01:01:37.440218+0000","last_unstale":"2024-04-25T01:27:49.439797+0000","last_undegraded":"2024-04-25T01:27:49.439797+0000","last_fullsized":"2024-04-25T01:27:49.439797+0000","mapping_epoch":23,"log_start":"27'3717","ondisk_log_start":"27'3717","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10011","last_scrub_stamp":"2024-04-25T01:27:49.439673+0000","last_deep_scrub":"27'10011","last_deep_scrub_stamp":"2024-04-25T01:27:49.439673+0000","last_clean_scrub_stamp":"2024-04-25T01:27:49.439673+0000","objects_scrubbed":0,"log_size":6294,"log_dups_size":0,"ondisk_log_size":6294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:02.785363+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":8890,"num_write":7291,"num_write_kb":1064218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9075","reported_seq":9342,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:48.698377+0000","last_change":"2024-04-25T01:27:48.698424+0000","last_active":"2024-04-25T01:27:48.698377+0000","last_peered":"2024-04-25T01:27:48.698377+0000","last_clean":"2024-04-25T01:27:48.698377+0000","last_became_active":"2024-04-25T01:01:37.443689+0000","last_became_peered":"2024-04-25T01:01:37.443689+0000","last_unstale":"2024-04-25T01:27:48.698377+0000","last_undegraded":"2024-04-25T01:27:48.698377+0000","last_fullsized":"2024-04-25T01:27:48.698377+0000","mapping_epoch":23,"log_start":"27'1719","ondisk_log_start":"27'1719","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9075","last_scrub_stamp":"2024-04-25T01:27:48.698252+0000","last_deep_scrub":"27'9075","last_deep_scrub_stamp":"2024-04-25T01:27:48.698252+0000","last_clean_scrub_stamp":"2024-04-25T01:27:48.698252+0000","objects_scrubbed":0,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:29.186057+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":227,"num_read_kb":57658,"num_write":6476,"num_write_kb":909388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2493","reported_seq":2536,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:33.126214+0000","last_change":"2024-04-25T01:27:33.126249+0000","last_active":"2024-04-25T01:27:33.126214+0000","last_peered":"2024-04-25T01:27:33.126214+0000","last_clean":"2024-04-25T01:27:33.126214+0000","last_became_active":"2024-04-25T01:01:33.393312+0000","last_became_peered":"2024-04-25T01:01:33.393312+0000","last_unstale":"2024-04-25T01:27:33.126214+0000","last_undegraded":"2024-04-25T01:27:33.126214+0000","last_fullsized":"2024-04-25T01:27:33.126214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2493","last_scrub_stamp":"2024-04-25T01:27:33.126122+0000","last_deep_scrub":"27'2493","last_deep_scrub_stamp":"2024-04-25T01:27:33.126122+0000","last_clean_scrub_stamp":"2024-04-25T01:27:33.126122+0000","objects_scrubbed":0,"log_size":2493,"log_dups_size":0,"ondisk_log_size":2493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:02.410784+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9900","reported_seq":9945,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.541982+0000","last_change":"2024-04-25T01:27:41.542011+0000","last_active":"2024-04-25T01:27:41.541982+0000","last_peered":"2024-04-25T01:27:41.541982+0000","last_clean":"2024-04-25T01:27:41.541982+0000","last_became_active":"2024-04-25T01:01:32.398076+0000","last_became_peered":"2024-04-25T01:01:32.398076+0000","last_unstale":"2024-04-25T01:27:41.541982+0000","last_undegraded":"2024-04-25T01:27:41.541982+0000","last_fullsized":"2024-04-25T01:27:41.541982+0000","mapping_epoch":18,"log_start":"27'2500","ondisk_log_start":"27'2500","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9900","last_scrub_stamp":"2024-04-25T01:27:41.541896+0000","last_deep_scrub":"27'9900","last_deep_scrub_stamp":"2024-04-25T01:27:41.541896+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.541896+0000","objects_scrubbed":1,"log_size":7400,"log_dups_size":0,"ondisk_log_size":7400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:45.494762+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":68120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8969","reported_seq":9837,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.907922+0000","last_change":"2024-04-25T01:27:45.907966+0000","last_active":"2024-04-25T01:27:45.907922+0000","last_peered":"2024-04-25T01:27:45.907922+0000","last_clean":"2024-04-25T01:27:45.907922+0000","last_became_active":"2024-04-25T01:01:37.434943+0000","last_became_peered":"2024-04-25T01:01:37.434943+0000","last_unstale":"2024-04-25T01:27:45.907922+0000","last_undegraded":"2024-04-25T01:27:45.907922+0000","last_fullsized":"2024-04-25T01:27:45.907922+0000","mapping_epoch":23,"log_start":"27'1000","ondisk_log_start":"27'1000","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8969","last_scrub_stamp":"2024-04-25T01:27:45.907792+0000","last_deep_scrub":"27'8969","last_deep_scrub_stamp":"2024-04-25T01:27:45.907792+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.907792+0000","objects_scrubbed":0,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:33.920958+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":829,"num_read_kb":210566,"num_write":6335,"num_write_kb":893076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2595","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.369360+0000","last_change":"2024-04-25T01:27:40.369401+0000","last_active":"2024-04-25T01:27:40.369360+0000","last_peered":"2024-04-25T01:27:40.369360+0000","last_clean":"2024-04-25T01:27:40.369360+0000","last_became_active":"2024-04-25T01:01:33.394717+0000","last_became_peered":"2024-04-25T01:01:33.394717+0000","last_unstale":"2024-04-25T01:27:40.369360+0000","last_undegraded":"2024-04-25T01:27:40.369360+0000","last_fullsized":"2024-04-25T01:27:40.369360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2595","last_scrub_stamp":"2024-04-25T01:27:40.369259+0000","last_deep_scrub":"27'2595","last_deep_scrub_stamp":"2024-04-25T01:27:40.369259+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.369259+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:19.795290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5346","reported_seq":5391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.415118+0000","last_change":"2024-04-25T01:27:37.415146+0000","last_active":"2024-04-25T01:27:37.415118+0000","last_peered":"2024-04-25T01:27:37.415118+0000","last_clean":"2024-04-25T01:27:37.415118+0000","last_became_active":"2024-04-25T01:01:32.391244+0000","last_became_peered":"2024-04-25T01:01:32.391244+0000","last_unstale":"2024-04-25T01:27:37.415118+0000","last_undegraded":"2024-04-25T01:27:37.415118+0000","last_fullsized":"2024-04-25T01:27:37.415118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5346","last_scrub_stamp":"2024-04-25T01:27:37.415049+0000","last_deep_scrub":"27'5346","last_deep_scrub_stamp":"2024-04-25T01:27:37.415049+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.415049+0000","objects_scrubbed":2,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:30.662191+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5344,"num_write_kb":44512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9503","reported_seq":9548,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.938558+0000","last_change":"2024-04-25T01:27:37.938589+0000","last_active":"2024-04-25T01:27:37.938558+0000","last_peered":"2024-04-25T01:27:37.938558+0000","last_clean":"2024-04-25T01:27:37.938558+0000","last_became_active":"2024-04-25T01:01:32.397709+0000","last_became_peered":"2024-04-25T01:01:32.397709+0000","last_unstale":"2024-04-25T01:27:37.938558+0000","last_undegraded":"2024-04-25T01:27:37.938558+0000","last_fullsized":"2024-04-25T01:27:37.938558+0000","mapping_epoch":18,"log_start":"27'1600","ondisk_log_start":"27'1600","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9503","last_scrub_stamp":"2024-04-25T01:27:37.938497+0000","last_deep_scrub":"27'9503","last_deep_scrub_stamp":"2024-04-25T01:27:37.938497+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.938497+0000","objects_scrubbed":1,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:13.626659+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":9513,"num_write_kb":71968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2531","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.688552+0000","last_change":"2024-04-25T01:27:41.688573+0000","last_active":"2024-04-25T01:27:41.688552+0000","last_peered":"2024-04-25T01:27:41.688552+0000","last_clean":"2024-04-25T01:27:41.688552+0000","last_became_active":"2024-04-25T01:01:33.406134+0000","last_became_peered":"2024-04-25T01:01:33.406134+0000","last_unstale":"2024-04-25T01:27:41.688552+0000","last_undegraded":"2024-04-25T01:27:41.688552+0000","last_fullsized":"2024-04-25T01:27:41.688552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2531","last_scrub_stamp":"2024-04-25T01:27:41.688476+0000","last_deep_scrub":"27'2531","last_deep_scrub_stamp":"2024-04-25T01:27:41.688476+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.688476+0000","objects_scrubbed":0,"log_size":2531,"log_dups_size":0,"ondisk_log_size":2531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:29.701642+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9470","reported_seq":9634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.002524+0000","last_change":"2024-04-25T01:27:39.002566+0000","last_active":"2024-04-25T01:27:39.002524+0000","last_peered":"2024-04-25T01:27:39.002524+0000","last_clean":"2024-04-25T01:27:39.002524+0000","last_became_active":"2024-04-25T01:01:37.437897+0000","last_became_peered":"2024-04-25T01:01:37.437897+0000","last_unstale":"2024-04-25T01:27:39.002524+0000","last_undegraded":"2024-04-25T01:27:39.002524+0000","last_fullsized":"2024-04-25T01:27:39.002524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9470","last_scrub_stamp":"2024-04-25T01:27:39.002428+0000","last_deep_scrub":"27'9470","last_deep_scrub_stamp":"2024-04-25T01:27:39.002428+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.002428+0000","objects_scrubbed":0,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:23:16.887433+0000","scrub_duration":56,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":32258,"num_write":6759,"num_write_kb":1001286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8026","reported_seq":8071,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:39.353936+0000","last_change":"2024-04-25T01:27:39.353959+0000","last_active":"2024-04-25T01:27:39.353936+0000","last_peered":"2024-04-25T01:27:39.353936+0000","last_clean":"2024-04-25T01:27:39.353936+0000","last_became_active":"2024-04-25T01:01:32.399895+0000","last_became_peered":"2024-04-25T01:01:32.399895+0000","last_unstale":"2024-04-25T01:27:39.353936+0000","last_undegraded":"2024-04-25T01:27:39.353936+0000","last_fullsized":"2024-04-25T01:27:39.353936+0000","mapping_epoch":18,"log_start":"27'1700","ondisk_log_start":"27'1700","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8026","last_scrub_stamp":"2024-04-25T01:27:39.353862+0000","last_deep_scrub":"27'8026","last_deep_scrub_stamp":"2024-04-25T01:27:39.353862+0000","last_clean_scrub_stamp":"2024-04-25T01:27:39.353862+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:39.246362+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8021,"num_write_kb":66584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2569","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.695418+0000","last_change":"2024-04-25T01:27:34.695454+0000","last_active":"2024-04-25T01:27:34.695418+0000","last_peered":"2024-04-25T01:27:34.695418+0000","last_clean":"2024-04-25T01:27:34.695418+0000","last_became_active":"2024-04-25T01:01:33.386894+0000","last_became_peered":"2024-04-25T01:01:33.386894+0000","last_unstale":"2024-04-25T01:27:34.695418+0000","last_undegraded":"2024-04-25T01:27:34.695418+0000","last_fullsized":"2024-04-25T01:27:34.695418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2569","last_scrub_stamp":"2024-04-25T01:27:34.695334+0000","last_deep_scrub":"27'2569","last_deep_scrub_stamp":"2024-04-25T01:27:34.695334+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.695334+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:30:03.979996+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9481","reported_seq":9886,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:48.400344+0000","last_change":"2024-04-25T01:27:48.400384+0000","last_active":"2024-04-25T01:27:48.400344+0000","last_peered":"2024-04-25T01:27:48.400344+0000","last_clean":"2024-04-25T01:27:48.400344+0000","last_became_active":"2024-04-25T01:01:37.439713+0000","last_became_peered":"2024-04-25T01:01:37.439713+0000","last_unstale":"2024-04-25T01:27:48.400344+0000","last_undegraded":"2024-04-25T01:27:48.400344+0000","last_fullsized":"2024-04-25T01:27:48.400344+0000","mapping_epoch":23,"log_start":"27'3225","ondisk_log_start":"27'3225","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9481","last_scrub_stamp":"2024-04-25T01:27:48.400244+0000","last_deep_scrub":"27'9481","last_deep_scrub_stamp":"2024-04-25T01:27:48.400244+0000","last_clean_scrub_stamp":"2024-04-25T01:27:48.400244+0000","objects_scrubbed":0,"log_size":6256,"log_dups_size":0,"ondisk_log_size":6256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:18:28.888049+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":368,"num_read_kb":93472,"num_write":6767,"num_write_kb":1052015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2469","reported_seq":2514,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.978713+0000","last_change":"2024-04-25T01:27:41.978750+0000","last_active":"2024-04-25T01:27:41.978713+0000","last_peered":"2024-04-25T01:27:41.978713+0000","last_clean":"2024-04-25T01:27:41.978713+0000","last_became_active":"2024-04-25T01:01:33.402501+0000","last_became_peered":"2024-04-25T01:01:33.402501+0000","last_unstale":"2024-04-25T01:27:41.978713+0000","last_undegraded":"2024-04-25T01:27:41.978713+0000","last_fullsized":"2024-04-25T01:27:41.978713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2469","last_scrub_stamp":"2024-04-25T01:27:41.978617+0000","last_deep_scrub":"27'2469","last_deep_scrub_stamp":"2024-04-25T01:27:41.978617+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.978617+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:59.683309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2598","reported_seq":2643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.939588+0000","last_change":"2024-04-25T01:27:36.939609+0000","last_active":"2024-04-25T01:27:36.939588+0000","last_peered":"2024-04-25T01:27:36.939588+0000","last_clean":"2024-04-25T01:27:36.939588+0000","last_became_active":"2024-04-25T01:01:33.400360+0000","last_became_peered":"2024-04-25T01:01:33.400360+0000","last_unstale":"2024-04-25T01:27:36.939588+0000","last_undegraded":"2024-04-25T01:27:36.939588+0000","last_fullsized":"2024-04-25T01:27:36.939588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2598","last_scrub_stamp":"2024-04-25T01:27:36.939526+0000","last_deep_scrub":"27'2598","last_deep_scrub_stamp":"2024-04-25T01:27:36.939526+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.939526+0000","objects_scrubbed":0,"log_size":2598,"log_dups_size":0,"ondisk_log_size":2598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:15.073790+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2500","reported_seq":2545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.457908+0000","last_change":"2024-04-25T01:27:46.457949+0000","last_active":"2024-04-25T01:27:46.457908+0000","last_peered":"2024-04-25T01:27:46.457908+0000","last_clean":"2024-04-25T01:27:46.457908+0000","last_became_active":"2024-04-25T01:01:33.411521+0000","last_became_peered":"2024-04-25T01:01:33.411521+0000","last_unstale":"2024-04-25T01:27:46.457908+0000","last_undegraded":"2024-04-25T01:27:46.457908+0000","last_fullsized":"2024-04-25T01:27:46.457908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2500","last_scrub_stamp":"2024-04-25T01:27:46.457808+0000","last_deep_scrub":"27'2500","last_deep_scrub_stamp":"2024-04-25T01:27:46.457808+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.457808+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:24.287520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2579","reported_seq":2624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.636022+0000","last_change":"2024-04-25T01:27:36.636043+0000","last_active":"2024-04-25T01:27:36.636022+0000","last_peered":"2024-04-25T01:27:36.636022+0000","last_clean":"2024-04-25T01:27:36.636022+0000","last_became_active":"2024-04-25T01:01:33.394497+0000","last_became_peered":"2024-04-25T01:01:33.394497+0000","last_unstale":"2024-04-25T01:27:36.636022+0000","last_undegraded":"2024-04-25T01:27:36.636022+0000","last_fullsized":"2024-04-25T01:27:36.636022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2579","last_scrub_stamp":"2024-04-25T01:27:36.635958+0000","last_deep_scrub":"27'2579","last_deep_scrub_stamp":"2024-04-25T01:27:36.635958+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.635958+0000","objects_scrubbed":0,"log_size":2579,"log_dups_size":0,"ondisk_log_size":2579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:43.575706+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2580","reported_seq":2625,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.367014+0000","last_change":"2024-04-25T01:27:36.367036+0000","last_active":"2024-04-25T01:27:36.367014+0000","last_peered":"2024-04-25T01:27:36.367014+0000","last_clean":"2024-04-25T01:27:36.367014+0000","last_became_active":"2024-04-25T01:01:33.404239+0000","last_became_peered":"2024-04-25T01:01:33.404239+0000","last_unstale":"2024-04-25T01:27:36.367014+0000","last_undegraded":"2024-04-25T01:27:36.367014+0000","last_fullsized":"2024-04-25T01:27:36.367014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2580","last_scrub_stamp":"2024-04-25T01:27:36.366951+0000","last_deep_scrub":"27'2580","last_deep_scrub_stamp":"2024-04-25T01:27:36.366951+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.366951+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:53:11.732345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2456","reported_seq":2501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:41.032185+0000","last_change":"2024-04-25T01:27:41.032226+0000","last_active":"2024-04-25T01:27:41.032185+0000","last_peered":"2024-04-25T01:27:41.032185+0000","last_clean":"2024-04-25T01:27:41.032185+0000","last_became_active":"2024-04-25T01:01:33.392598+0000","last_became_peered":"2024-04-25T01:01:33.392598+0000","last_unstale":"2024-04-25T01:27:41.032185+0000","last_undegraded":"2024-04-25T01:27:41.032185+0000","last_fullsized":"2024-04-25T01:27:41.032185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2456","last_scrub_stamp":"2024-04-25T01:27:41.032060+0000","last_deep_scrub":"27'2456","last_deep_scrub_stamp":"2024-04-25T01:27:41.032060+0000","last_clean_scrub_stamp":"2024-04-25T01:27:41.032060+0000","objects_scrubbed":0,"log_size":2456,"log_dups_size":0,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:43.211759+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2589","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.927784+0000","last_change":"2024-04-25T01:27:43.927805+0000","last_active":"2024-04-25T01:27:43.927784+0000","last_peered":"2024-04-25T01:27:43.927784+0000","last_clean":"2024-04-25T01:27:43.927784+0000","last_became_active":"2024-04-25T01:01:33.404661+0000","last_became_peered":"2024-04-25T01:01:33.404661+0000","last_unstale":"2024-04-25T01:27:43.927784+0000","last_undegraded":"2024-04-25T01:27:43.927784+0000","last_fullsized":"2024-04-25T01:27:43.927784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2589","last_scrub_stamp":"2024-04-25T01:27:43.927712+0000","last_deep_scrub":"27'2589","last_deep_scrub_stamp":"2024-04-25T01:27:43.927712+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.927712+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:43.820212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2556","reported_seq":2601,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.424875+0000","last_change":"2024-04-25T01:27:35.424875+0000","last_active":"2024-04-25T01:27:35.424875+0000","last_peered":"2024-04-25T01:27:35.424875+0000","last_clean":"2024-04-25T01:27:35.424875+0000","last_became_active":"2024-04-25T01:01:33.405219+0000","last_became_peered":"2024-04-25T01:01:33.405219+0000","last_unstale":"2024-04-25T01:27:35.424875+0000","last_undegraded":"2024-04-25T01:27:35.424875+0000","last_fullsized":"2024-04-25T01:27:35.424875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2556","last_scrub_stamp":"2024-04-25T01:27:35.424673+0000","last_deep_scrub":"27'2556","last_deep_scrub_stamp":"2024-04-25T01:27:35.424673+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.424673+0000","objects_scrubbed":0,"log_size":2556,"log_dups_size":0,"ondisk_log_size":2556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:37.740233+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2533","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.518201+0000","last_change":"2024-04-25T01:27:43.518248+0000","last_active":"2024-04-25T01:27:43.518201+0000","last_peered":"2024-04-25T01:27:43.518201+0000","last_clean":"2024-04-25T01:27:43.518201+0000","last_became_active":"2024-04-25T01:01:33.401917+0000","last_became_peered":"2024-04-25T01:01:33.401917+0000","last_unstale":"2024-04-25T01:27:43.518201+0000","last_undegraded":"2024-04-25T01:27:43.518201+0000","last_fullsized":"2024-04-25T01:27:43.518201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2533","last_scrub_stamp":"2024-04-25T01:27:43.518124+0000","last_deep_scrub":"27'2533","last_deep_scrub_stamp":"2024-04-25T01:27:43.518124+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.518124+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:39.730744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'2519","reported_seq":2563,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:33.345264+0000","last_change":"2024-04-25T01:27:33.345287+0000","last_active":"2024-04-25T01:27:33.345264+0000","last_peered":"2024-04-25T01:27:33.345264+0000","last_clean":"2024-04-25T01:27:33.345264+0000","last_became_active":"2024-04-25T01:01:33.404988+0000","last_became_peered":"2024-04-25T01:01:33.404988+0000","last_unstale":"2024-04-25T01:27:33.345264+0000","last_undegraded":"2024-04-25T01:27:33.345264+0000","last_fullsized":"2024-04-25T01:27:33.345264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2519","last_scrub_stamp":"2024-04-25T01:27:33.345187+0000","last_deep_scrub":"27'2519","last_deep_scrub_stamp":"2024-04-25T01:27:33.345187+0000","last_clean_scrub_stamp":"2024-04-25T01:27:33.345187+0000","objects_scrubbed":0,"log_size":2519,"log_dups_size":0,"ondisk_log_size":2519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:19.300716+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2560","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:50.473561+0000","last_change":"2024-04-25T01:27:50.473584+0000","last_active":"2024-04-25T01:27:50.473561+0000","last_peered":"2024-04-25T01:27:50.473561+0000","last_clean":"2024-04-25T01:27:50.473561+0000","last_became_active":"2024-04-25T01:01:33.413078+0000","last_became_peered":"2024-04-25T01:01:33.413078+0000","last_unstale":"2024-04-25T01:27:50.473561+0000","last_undegraded":"2024-04-25T01:27:50.473561+0000","last_fullsized":"2024-04-25T01:27:50.473561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2560","last_scrub_stamp":"2024-04-25T01:27:50.473491+0000","last_deep_scrub":"27'2560","last_deep_scrub_stamp":"2024-04-25T01:27:50.473491+0000","last_clean_scrub_stamp":"2024-04-25T01:27:50.473491+0000","objects_scrubbed":0,"log_size":2560,"log_dups_size":0,"ondisk_log_size":2560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:27.533922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2589","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:32.612966+0000","last_change":"2024-04-25T01:27:32.613002+0000","last_active":"2024-04-25T01:27:32.612966+0000","last_peered":"2024-04-25T01:27:32.612966+0000","last_clean":"2024-04-25T01:27:32.612966+0000","last_became_active":"2024-04-25T01:01:33.404043+0000","last_became_peered":"2024-04-25T01:01:33.404043+0000","last_unstale":"2024-04-25T01:27:32.612966+0000","last_undegraded":"2024-04-25T01:27:32.612966+0000","last_fullsized":"2024-04-25T01:27:32.612966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2589","last_scrub_stamp":"2024-04-25T01:27:32.612876+0000","last_deep_scrub":"27'2589","last_deep_scrub_stamp":"2024-04-25T01:27:32.612876+0000","last_clean_scrub_stamp":"2024-04-25T01:27:32.612876+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:47:55.831369+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2586","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:42.966283+0000","last_change":"2024-04-25T01:27:42.966303+0000","last_active":"2024-04-25T01:27:42.966283+0000","last_peered":"2024-04-25T01:27:42.966283+0000","last_clean":"2024-04-25T01:27:42.966283+0000","last_became_active":"2024-04-25T01:01:33.402977+0000","last_became_peered":"2024-04-25T01:01:33.402977+0000","last_unstale":"2024-04-25T01:27:42.966283+0000","last_undegraded":"2024-04-25T01:27:42.966283+0000","last_fullsized":"2024-04-25T01:27:42.966283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2586","last_scrub_stamp":"2024-04-25T01:27:42.966212+0000","last_deep_scrub":"27'2586","last_deep_scrub_stamp":"2024-04-25T01:27:42.966212+0000","last_clean_scrub_stamp":"2024-04-25T01:27:42.966212+0000","objects_scrubbed":0,"log_size":2586,"log_dups_size":0,"ondisk_log_size":2586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:47.598056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2585","reported_seq":2629,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.277932+0000","last_change":"2024-04-25T01:27:46.277977+0000","last_active":"2024-04-25T01:27:46.277932+0000","last_peered":"2024-04-25T01:27:46.277932+0000","last_clean":"2024-04-25T01:27:46.277932+0000","last_became_active":"2024-04-25T01:01:33.395239+0000","last_became_peered":"2024-04-25T01:01:33.395239+0000","last_unstale":"2024-04-25T01:27:46.277932+0000","last_undegraded":"2024-04-25T01:27:46.277932+0000","last_fullsized":"2024-04-25T01:27:46.277932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2585","last_scrub_stamp":"2024-04-25T01:27:46.277834+0000","last_deep_scrub":"27'2585","last_deep_scrub_stamp":"2024-04-25T01:27:46.277834+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.277834+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:26.599006+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2524","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.958724+0000","last_change":"2024-04-25T01:27:37.958744+0000","last_active":"2024-04-25T01:27:37.958724+0000","last_peered":"2024-04-25T01:27:37.958724+0000","last_clean":"2024-04-25T01:27:37.958724+0000","last_became_active":"2024-04-25T01:01:33.400578+0000","last_became_peered":"2024-04-25T01:01:33.400578+0000","last_unstale":"2024-04-25T01:27:37.958724+0000","last_undegraded":"2024-04-25T01:27:37.958724+0000","last_fullsized":"2024-04-25T01:27:37.958724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2524","last_scrub_stamp":"2024-04-25T01:27:37.958667+0000","last_deep_scrub":"27'2524","last_deep_scrub_stamp":"2024-04-25T01:27:37.958667+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.958667+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:30.266653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2594","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:36.524422+0000","last_change":"2024-04-25T01:27:36.524463+0000","last_active":"2024-04-25T01:27:36.524422+0000","last_peered":"2024-04-25T01:27:36.524422+0000","last_clean":"2024-04-25T01:27:36.524422+0000","last_became_active":"2024-04-25T01:01:33.408876+0000","last_became_peered":"2024-04-25T01:01:33.408876+0000","last_unstale":"2024-04-25T01:27:36.524422+0000","last_undegraded":"2024-04-25T01:27:36.524422+0000","last_fullsized":"2024-04-25T01:27:36.524422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2594","last_scrub_stamp":"2024-04-25T01:27:36.524309+0000","last_deep_scrub":"27'2594","last_deep_scrub_stamp":"2024-04-25T01:27:36.524309+0000","last_clean_scrub_stamp":"2024-04-25T01:27:36.524309+0000","objects_scrubbed":0,"log_size":2594,"log_dups_size":0,"ondisk_log_size":2594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:33.654915+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2516","reported_seq":2560,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:42.475473+0000","last_change":"2024-04-25T01:27:42.475495+0000","last_active":"2024-04-25T01:27:42.475473+0000","last_peered":"2024-04-25T01:27:42.475473+0000","last_clean":"2024-04-25T01:27:42.475473+0000","last_became_active":"2024-04-25T01:01:33.402387+0000","last_became_peered":"2024-04-25T01:01:33.402387+0000","last_unstale":"2024-04-25T01:27:42.475473+0000","last_undegraded":"2024-04-25T01:27:42.475473+0000","last_fullsized":"2024-04-25T01:27:42.475473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2516","last_scrub_stamp":"2024-04-25T01:27:42.475409+0000","last_deep_scrub":"27'2516","last_deep_scrub_stamp":"2024-04-25T01:27:42.475409+0000","last_clean_scrub_stamp":"2024-04-25T01:27:42.475409+0000","objects_scrubbed":0,"log_size":2516,"log_dups_size":0,"ondisk_log_size":2516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:13.492158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2645","reported_seq":2689,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:40.059303+0000","last_change":"2024-04-25T01:27:40.059327+0000","last_active":"2024-04-25T01:27:40.059303+0000","last_peered":"2024-04-25T01:27:40.059303+0000","last_clean":"2024-04-25T01:27:40.059303+0000","last_became_active":"2024-04-25T01:01:33.388482+0000","last_became_peered":"2024-04-25T01:01:33.388482+0000","last_unstale":"2024-04-25T01:27:40.059303+0000","last_undegraded":"2024-04-25T01:27:40.059303+0000","last_fullsized":"2024-04-25T01:27:40.059303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2645","last_scrub_stamp":"2024-04-25T01:27:40.059221+0000","last_deep_scrub":"27'2645","last_deep_scrub_stamp":"2024-04-25T01:27:40.059221+0000","last_clean_scrub_stamp":"2024-04-25T01:27:40.059221+0000","objects_scrubbed":0,"log_size":2645,"log_dups_size":0,"ondisk_log_size":2645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:33.306363+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2495","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:37.661949+0000","last_change":"2024-04-25T01:27:37.661971+0000","last_active":"2024-04-25T01:27:37.661949+0000","last_peered":"2024-04-25T01:27:37.661949+0000","last_clean":"2024-04-25T01:27:37.661949+0000","last_became_active":"2024-04-25T01:01:33.398771+0000","last_became_peered":"2024-04-25T01:01:33.398771+0000","last_unstale":"2024-04-25T01:27:37.661949+0000","last_undegraded":"2024-04-25T01:27:37.661949+0000","last_fullsized":"2024-04-25T01:27:37.661949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2495","last_scrub_stamp":"2024-04-25T01:27:37.661863+0000","last_deep_scrub":"27'2495","last_deep_scrub_stamp":"2024-04-25T01:27:37.661863+0000","last_clean_scrub_stamp":"2024-04-25T01:27:37.661863+0000","objects_scrubbed":0,"log_size":2495,"log_dups_size":0,"ondisk_log_size":2495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:55.366555+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2489","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:34.402567+0000","last_change":"2024-04-25T01:27:34.402590+0000","last_active":"2024-04-25T01:27:34.402567+0000","last_peered":"2024-04-25T01:27:34.402567+0000","last_clean":"2024-04-25T01:27:34.402567+0000","last_became_active":"2024-04-25T01:01:33.404648+0000","last_became_peered":"2024-04-25T01:01:33.404648+0000","last_unstale":"2024-04-25T01:27:34.402567+0000","last_undegraded":"2024-04-25T01:27:34.402567+0000","last_fullsized":"2024-04-25T01:27:34.402567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2489","last_scrub_stamp":"2024-04-25T01:27:34.402474+0000","last_deep_scrub":"27'2489","last_deep_scrub_stamp":"2024-04-25T01:27:34.402474+0000","last_clean_scrub_stamp":"2024-04-25T01:27:34.402474+0000","objects_scrubbed":0,"log_size":2489,"log_dups_size":0,"ondisk_log_size":2489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:24.045721+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2444","reported_seq":2487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:38.727465+0000","last_change":"2024-04-25T01:27:38.727502+0000","last_active":"2024-04-25T01:27:38.727465+0000","last_peered":"2024-04-25T01:27:38.727465+0000","last_clean":"2024-04-25T01:27:38.727465+0000","last_became_active":"2024-04-25T01:01:33.406152+0000","last_became_peered":"2024-04-25T01:01:33.406152+0000","last_unstale":"2024-04-25T01:27:38.727465+0000","last_undegraded":"2024-04-25T01:27:38.727465+0000","last_fullsized":"2024-04-25T01:27:38.727465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2444","last_scrub_stamp":"2024-04-25T01:27:38.727398+0000","last_deep_scrub":"27'2444","last_deep_scrub_stamp":"2024-04-25T01:27:38.727398+0000","last_clean_scrub_stamp":"2024-04-25T01:27:38.727398+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:28.370645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2476","reported_seq":2524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:46.858623+0000","last_change":"2024-04-25T01:27:46.858645+0000","last_active":"2024-04-25T01:27:46.858623+0000","last_peered":"2024-04-25T01:27:46.858623+0000","last_clean":"2024-04-25T01:27:46.858623+0000","last_became_active":"2024-04-25T01:02:13.748901+0000","last_became_peered":"2024-04-25T01:02:13.748901+0000","last_unstale":"2024-04-25T01:27:46.858623+0000","last_undegraded":"2024-04-25T01:27:46.858623+0000","last_fullsized":"2024-04-25T01:27:46.858623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2476","last_scrub_stamp":"2024-04-25T01:27:46.858543+0000","last_deep_scrub":"27'2476","last_deep_scrub_stamp":"2024-04-25T01:27:46.858543+0000","last_clean_scrub_stamp":"2024-04-25T01:27:46.858543+0000","objects_scrubbed":0,"log_size":2476,"log_dups_size":0,"ondisk_log_size":2476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:05:28.251197+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2612","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.791615+0000","last_change":"2024-04-25T01:27:47.791637+0000","last_active":"2024-04-25T01:27:47.791615+0000","last_peered":"2024-04-25T01:27:47.791615+0000","last_clean":"2024-04-25T01:27:47.791615+0000","last_became_active":"2024-04-25T01:02:13.749859+0000","last_became_peered":"2024-04-25T01:02:13.749859+0000","last_unstale":"2024-04-25T01:27:47.791615+0000","last_undegraded":"2024-04-25T01:27:47.791615+0000","last_fullsized":"2024-04-25T01:27:47.791615+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2612","last_scrub_stamp":"2024-04-25T01:27:47.791546+0000","last_deep_scrub":"27'2612","last_deep_scrub_stamp":"2024-04-25T01:27:47.791546+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.791546+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:07.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2575","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:51.290965+0000","last_change":"2024-04-25T01:27:51.290996+0000","last_active":"2024-04-25T01:27:51.290965+0000","last_peered":"2024-04-25T01:27:51.290965+0000","last_clean":"2024-04-25T01:27:51.290965+0000","last_became_active":"2024-04-25T01:01:33.411331+0000","last_became_peered":"2024-04-25T01:01:33.411331+0000","last_unstale":"2024-04-25T01:27:51.290965+0000","last_undegraded":"2024-04-25T01:27:51.290965+0000","last_fullsized":"2024-04-25T01:27:51.290965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2575","last_scrub_stamp":"2024-04-25T01:27:51.290881+0000","last_deep_scrub":"27'2575","last_deep_scrub_stamp":"2024-04-25T01:27:51.290881+0000","last_clean_scrub_stamp":"2024-04-25T01:27:51.290881+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:21.411584+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2520","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:49.073385+0000","last_change":"2024-04-25T01:27:49.073406+0000","last_active":"2024-04-25T01:27:49.073385+0000","last_peered":"2024-04-25T01:27:49.073385+0000","last_clean":"2024-04-25T01:27:49.073385+0000","last_became_active":"2024-04-25T01:02:13.828599+0000","last_became_peered":"2024-04-25T01:02:13.828599+0000","last_unstale":"2024-04-25T01:27:49.073385+0000","last_undegraded":"2024-04-25T01:27:49.073385+0000","last_fullsized":"2024-04-25T01:27:49.073385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2520","last_scrub_stamp":"2024-04-25T01:27:49.073316+0000","last_deep_scrub":"27'2520","last_deep_scrub_stamp":"2024-04-25T01:27:49.073316+0000","last_clean_scrub_stamp":"2024-04-25T01:27:49.073316+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:02.591171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.39","version":"27'2527","reported_seq":2570,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:48.287425+0000","last_change":"2024-04-25T01:27:48.287448+0000","last_active":"2024-04-25T01:27:48.287425+0000","last_peered":"2024-04-25T01:27:48.287425+0000","last_clean":"2024-04-25T01:27:48.287425+0000","last_became_active":"2024-04-25T01:01:33.392643+0000","last_became_peered":"2024-04-25T01:01:33.392643+0000","last_unstale":"2024-04-25T01:27:48.287425+0000","last_undegraded":"2024-04-25T01:27:48.287425+0000","last_fullsized":"2024-04-25T01:27:48.287425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2527","last_scrub_stamp":"2024-04-25T01:27:48.287362+0000","last_deep_scrub":"27'2527","last_deep_scrub_stamp":"2024-04-25T01:27:48.287362+0000","last_clean_scrub_stamp":"2024-04-25T01:27:48.287362+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:26.117075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2472","reported_seq":2515,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:48.454457+0000","last_change":"2024-04-25T01:27:48.454501+0000","last_active":"2024-04-25T01:27:48.454457+0000","last_peered":"2024-04-25T01:27:48.454457+0000","last_clean":"2024-04-25T01:27:48.454457+0000","last_became_active":"2024-04-25T01:01:33.412527+0000","last_became_peered":"2024-04-25T01:01:33.412527+0000","last_unstale":"2024-04-25T01:27:48.454457+0000","last_undegraded":"2024-04-25T01:27:48.454457+0000","last_fullsized":"2024-04-25T01:27:48.454457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2472","last_scrub_stamp":"2024-04-25T01:27:48.454374+0000","last_deep_scrub":"27'2472","last_deep_scrub_stamp":"2024-04-25T01:27:48.454374+0000","last_clean_scrub_stamp":"2024-04-25T01:27:48.454374+0000","objects_scrubbed":0,"log_size":2472,"log_dups_size":0,"ondisk_log_size":2472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:28.931849+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2510","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:45.317270+0000","last_change":"2024-04-25T01:27:45.317312+0000","last_active":"2024-04-25T01:27:45.317270+0000","last_peered":"2024-04-25T01:27:45.317270+0000","last_clean":"2024-04-25T01:27:45.317270+0000","last_became_active":"2024-04-25T01:01:33.394732+0000","last_became_peered":"2024-04-25T01:01:33.394732+0000","last_unstale":"2024-04-25T01:27:45.317270+0000","last_undegraded":"2024-04-25T01:27:45.317270+0000","last_fullsized":"2024-04-25T01:27:45.317270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2510","last_scrub_stamp":"2024-04-25T01:27:45.317141+0000","last_deep_scrub":"27'2510","last_deep_scrub_stamp":"2024-04-25T01:27:45.317141+0000","last_clean_scrub_stamp":"2024-04-25T01:27:45.317141+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:41.529761+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2563","reported_seq":2609,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:47.433444+0000","last_change":"2024-04-25T01:27:47.433464+0000","last_active":"2024-04-25T01:27:47.433444+0000","last_peered":"2024-04-25T01:27:47.433444+0000","last_clean":"2024-04-25T01:27:47.433444+0000","last_became_active":"2024-04-25T01:01:33.412196+0000","last_became_peered":"2024-04-25T01:01:33.412196+0000","last_unstale":"2024-04-25T01:27:47.433444+0000","last_undegraded":"2024-04-25T01:27:47.433444+0000","last_fullsized":"2024-04-25T01:27:47.433444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2563","last_scrub_stamp":"2024-04-25T01:27:47.433379+0000","last_deep_scrub":"27'2563","last_deep_scrub_stamp":"2024-04-25T01:27:47.433379+0000","last_clean_scrub_stamp":"2024-04-25T01:27:47.433379+0000","objects_scrubbed":0,"log_size":2563,"log_dups_size":0,"ondisk_log_size":2563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:44.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2630","reported_seq":2675,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:35.631126+0000","last_change":"2024-04-25T01:27:35.631147+0000","last_active":"2024-04-25T01:27:35.631126+0000","last_peered":"2024-04-25T01:27:35.631126+0000","last_clean":"2024-04-25T01:27:35.631126+0000","last_became_active":"2024-04-25T01:01:33.393959+0000","last_became_peered":"2024-04-25T01:01:33.393959+0000","last_unstale":"2024-04-25T01:27:35.631126+0000","last_undegraded":"2024-04-25T01:27:35.631126+0000","last_fullsized":"2024-04-25T01:27:35.631126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2630","last_scrub_stamp":"2024-04-25T01:27:35.631063+0000","last_deep_scrub":"27'2630","last_deep_scrub_stamp":"2024-04-25T01:27:35.631063+0000","last_clean_scrub_stamp":"2024-04-25T01:27:35.631063+0000","objects_scrubbed":0,"log_size":2630,"log_dups_size":0,"ondisk_log_size":2630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:25.274311+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2469","reported_seq":2514,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:43.007873+0000","last_change":"2024-04-25T01:27:43.007904+0000","last_active":"2024-04-25T01:27:43.007873+0000","last_peered":"2024-04-25T01:27:43.007873+0000","last_clean":"2024-04-25T01:27:43.007873+0000","last_became_active":"2024-04-25T01:01:33.394935+0000","last_became_peered":"2024-04-25T01:01:33.394935+0000","last_unstale":"2024-04-25T01:27:43.007873+0000","last_undegraded":"2024-04-25T01:27:43.007873+0000","last_fullsized":"2024-04-25T01:27:43.007873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2469","last_scrub_stamp":"2024-04-25T01:27:43.007789+0000","last_deep_scrub":"27'2469","last_deep_scrub_stamp":"2024-04-25T01:27:43.007789+0000","last_clean_scrub_stamp":"2024-04-25T01:27:43.007789+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:10:01.086146+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2444","reported_seq":2489,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T01:27:49.287776+0000","last_change":"2024-04-25T01:27:49.287800+0000","last_active":"2024-04-25T01:27:49.287776+0000","last_peered":"2024-04-25T01:27:49.287776+0000","last_clean":"2024-04-25T01:27:49.287776+0000","last_became_active":"2024-04-25T01:01:33.398650+0000","last_became_peered":"2024-04-25T01:01:33.398650+0000","last_unstale":"2024-04-25T01:27:49.287776+0000","last_undegraded":"2024-04-25T01:27:49.287776+0000","last_fullsized":"2024-04-25T01:27:49.287776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2444","last_scrub_stamp":"2024-04-25T01:27:49.287691+0000","last_deep_scrub":"27'2444","last_deep_scrub_stamp":"2024-04-25T01:27:49.287691+0000","last_clean_scrub_stamp":"2024-04-25T01:27:49.287691+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:46.689362+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12020,"num_read_kb":3053080,"num_write":212634,"num_write_kb":30684192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232047,"ondisk_log_size":232047,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162605,"ondisk_log_size":162605,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":230496055,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":3112,"num_read_kb":19119,"num_write":209301,"num_write_kb":1676229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5927,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":461053952,"data_stored":460992110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194247,"ondisk_log_size":194247,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575170,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":948336,"kb_used_data":46696,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93423504,"statfs":{"total":96636764160,"available":95665668096,"internally_reserved":0,"allocated":47816704,"data_stored":47635630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":923269578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575167,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180336,"kb_used_data":66600,"kb_used_omap":1,"kb_used_meta":1113726,"kb_avail":93191504,"statfs":{"total":96636764160,"available":95428100096,"internally_reserved":0,"allocated":68198400,"data_stored":68016872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1140455878},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575168,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":991788,"kb_used_data":40292,"kb_used_omap":1,"kb_used_meta":951486,"kb_avail":93380052,"statfs":{"total":96636764160,"available":95621173248,"internally_reserved":0,"allocated":41259008,"data_stored":41081723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":974322117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575167,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1003440,"kb_used_data":65832,"kb_used_omap":1,"kb_used_meta":937598,"kb_avail":93368400,"statfs":{"total":96636764160,"available":95609241600,"internally_reserved":0,"allocated":67411968,"data_stored":67228022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960100805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575170,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1125612,"kb_used_data":65828,"kb_used_omap":1,"kb_used_meta":1059774,"kb_avail":93246228,"statfs":{"total":96636764160,"available":95484137472,"internally_reserved":0,"allocated":67407872,"data_stored":67228568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1085209031},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676068,"kb_used_data":53532,"kb_used_omap":1,"kb_used_meta":622526,"kb_avail":93695772,"statfs":{"total":96636764160,"available":95944470528,"internally_reserved":0,"allocated":54816768,"data_stored":54644984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637467077},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575169,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1237104,"kb_used_data":60776,"kb_used_omap":1,"kb_used_meta":1176318,"kb_avail":93134736,"statfs":{"total":96636764160,"available":95369969664,"internally_reserved":0,"allocated":62234624,"data_stored":62053313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1204550085},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575169,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820980,"kb_used_data":54124,"kb_used_omap":1,"kb_used_meta":766846,"kb_avail":93550860,"statfs":{"total":96636764160,"available":95796080640,"internally_reserved":0,"allocated":55422976,"data_stored":55235990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":785250757},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54538240,"data_stored":54526590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61943808,"data_stored":61934281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54525952,"data_stored":54525952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40968192,"data_stored":40962691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":67907584,"data_stored":67897840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":46931968,"data_stored":46926230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:27:55.664 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 Processor -- start 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 -- start start 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c40c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd7c40c9f70 0x7fd7c415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c40c5c10 con 0x7fd7c4159630 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c40c5650 con 0x7fd7c40c9f70 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.816+0000 7fd7ca11d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c40c5930 con 0x7fd7c40c8cc0 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c40c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c3fff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd7c40c9f70 0x7fd7c415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c40c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:57742/0 (socket says 172.21.15.129:57742) 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 -- 172.21.15.129:0/1237296753 learned_addr learned my addr 172.21.15.129:0/1237296753 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 -- 172.21.15.129:0/1237296753 >> v2:172.21.15.171:3300/0 conn(0x7fd7c40c9f70 msgr2=0x7fd7c415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 --2- 172.21.15.129:0/1237296753 >> v2:172.21.15.171:3300/0 conn(0x7fd7c40c9f70 0x7fd7c415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 -- 172.21.15.129:0/1237296753 >> v1:172.21.15.171:6789/0 conn(0x7fd7c4159630 legacy=0x7fd7c4156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c27fc640 1 -- 172.21.15.129:0/1237296753 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845267090 0 0) 0x7fd7c40c5c10 con 0x7fd7c4159630 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c37fe640 1 -- 172.21.15.129:0/1237296753 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c4155850 con 0x7fd7c40c8cc0 2024-04-25T01:27:55.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.817+0000 7fd7c3fff640 1 --2- 172.21.15.129:0/1237296753 >> v2:172.21.15.171:3300/0 conn(0x7fd7c40c9f70 0x7fd7c415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:55.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7c37fe640 1 --2- 172.21.15.129:0/1237296753 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c40c90a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009870 tx=0x7fd7b402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb310af614cf1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:55.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7c27fc640 1 -- 172.21.15.129:0/1237296753 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4037070 con 0x7fd7c40c8cc0 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7c27fc640 1 -- 172.21.15.129:0/1237296753 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4032440 con 0x7fd7c40c8cc0 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7c27fc640 1 -- 172.21.15.129:0/1237296753 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4032810 con 0x7fd7c40c8cc0 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 -- 172.21.15.129:0/1237296753 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 msgr2=0x7fd7c40c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/1237296753 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c40c90a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009870 tx=0x7fd7b402a1c0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 -- 172.21.15.129:0/1237296753 shutdown_connections 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/1237296753 >> v2:172.21.15.171:3300/0 conn(0x7fd7c40c9f70 0x7fd7c415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/1237296753 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c40c90a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 -- 172.21.15.129:0/1237296753 >> 172.21.15.129:0/1237296753 conn(0x7fd7c406c400 msgr2=0x7fd7c406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 -- 172.21.15.129:0/1237296753 shutdown_connections 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.818+0000 7fd7ca11d640 1 -- 172.21.15.129:0/1237296753 wait complete. 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.819+0000 7fd7ca11d640 1 Processor -- start 2024-04-25T01:27:55.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 -- start start 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c4153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7c40c9f70 0x7fd7c4157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 0x7fd7c4145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c406d1f0 con 0x7fd7c4159630 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c40ca400 con 0x7fd7c40c9f70 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7ca11d640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c40c5f30 con 0x7fd7c40c8cc0 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 0x7fd7c4145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 0x7fd7c4145b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56926/0 (socket says 172.21.15.129:56926) 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7c3fff640 1 -- 172.21.15.129:0/312201657 learned_addr learned my addr 172.21.15.129:0/312201657 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7c2ffd640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7c40c9f70 0x7fd7c4157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.820+0000 7fd7c37fe640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c4153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c2ffd640 1 -- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 msgr2=0x7fd7c4153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c2ffd640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c4153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c2ffd640 1 -- 172.21.15.129:0/312201657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 msgr2=0x7fd7c4145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c2ffd640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 0x7fd7c4145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c2ffd640 1 -- 172.21.15.129:0/312201657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c4154400 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c3fff640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 0x7fd7c4145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c37fe640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c4153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7c2ffd640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7c40c9f70 0x7fd7c4157ce0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c450 tx=0x7fd7b800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8017070 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b80035f0 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c4154fb0 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80039e0 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.821+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c4155570 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.822+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c4075550 con 0x7fd7c40c9f70 2024-04-25T01:27:55.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.823+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7b801a070 con 0x7fd7c40c9f70 2024-04-25T01:27:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.823+0000 7fd7abfff640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd7b006be30 0x7fd7b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.823+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd7b808a520 con 0x7fd7c40c9f70 2024-04-25T01:27:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.823+0000 7fd7c37fe640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd7b006be30 0x7fd7b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:55.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.824+0000 7fd7c37fe640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd7b006be30 0x7fd7b006e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4034000 tx=0x7fd7b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:55.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:55.828+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7c4154400 con 0x7fd7c40c9f70 2024-04-25T01:27:55.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.009+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fd7c40cb930 con 0x7fd7c40c9f70 2024-04-25T01:27:55.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.014+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd7b8058510 con 0x7fd7c40c9f70 2024-04-25T01:27:55.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.014+0000 7fd7abfff640 1 -- 172.21.15.129:0/312201657 <== mon.1 v2:172.21.15.171:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fd7b805c810 con 0x7fd7c40c9f70 2024-04-25T01:27:55.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd7b006be30 msgr2=0x7fd7b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd7b006be30 0x7fd7b006e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4034000 tx=0x7fd7b402e000 comp rx=0 tx=0).stop 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7c40c9f70 msgr2=0x7fd7c4157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7c40c9f70 0x7fd7c4157ce0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c450 tx=0x7fd7b800c910 comp rx=0 tx=0).stop 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 shutdown_connections 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd7b006be30 0x7fd7b006e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7c4159630 0x7fd7c4145b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7c40c9f70 0x7fd7c4157ce0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 --2- 172.21.15.129:0/312201657 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd7c40c8cc0 0x7fd7c4153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 >> 172.21.15.129:0/312201657 conn(0x7fd7c406c400 msgr2=0x7fd7c40c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 shutdown_connections 2024-04-25T01:27:55.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.017+0000 7fd7ca11d640 1 -- 172.21.15.129:0/312201657 wait complete. 2024-04-25T01:27:56.010 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T01:27:56.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:27:56.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.129+0000 7fc3f7609640 1 Processor -- start 2024-04-25T01:27:56.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f7609640 1 -- start start 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f7609640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc3f0152fa0 0x7fc3f00ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f7609640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f7609640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f00c9860 con 0x7fc3f00c5520 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f7609640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f00c5a70 con 0x7fc3f0152fa0 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f7609640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f00c5d50 con 0x7fc3f01561e0 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f53a5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc3f0152fa0 0x7fc3f00ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f5ba6640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.130+0000 7fc3f4ba4640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fc3f00c5520 0x7fc3f00c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:57658/0 (socket says 172.21.15.129:57658) 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.131+0000 7fc3f4ba4640 1 -- 172.21.15.129:0/2486672122 learned_addr learned my addr 172.21.15.129:0/2486672122 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.131+0000 7fc3f5ba6640 1 -- 172.21.15.129:0/2486672122 >> v2:172.21.15.171:3300/0 conn(0x7fc3f0152fa0 msgr2=0x7fc3f00ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:56.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.131+0000 7fc3f5ba6640 1 --2- 172.21.15.129:0/2486672122 >> v2:172.21.15.171:3300/0 conn(0x7fc3f0152fa0 0x7fc3f00ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.131+0000 7fc3f5ba6640 1 -- 172.21.15.129:0/2486672122 >> v1:172.21.15.171:6789/0 conn(0x7fc3f00c5520 legacy=0x7fc3f00c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.131+0000 7fc3f5ba6640 1 -- 172.21.15.129:0/2486672122 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f00c7650 con 0x7fc3f01561e0 2024-04-25T01:27:56.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.131+0000 7fc3f53a5640 1 --2- 172.21.15.129:0/2486672122 >> v2:172.21.15.171:3300/0 conn(0x7fc3f0152fa0 0x7fc3f00ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:56.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f5ba6640 1 --2- 172.21.15.129:0/2486672122 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f00c8cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009d90 tx=0x7fc3e802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ef7775cf5cfedb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3e7fff640 1 -- 172.21.15.129:0/2486672122 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007740 con 0x7fc3f01561e0 2024-04-25T01:27:56.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3e7fff640 1 -- 172.21.15.129:0/2486672122 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc3e80078e0 con 0x7fc3f01561e0 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3e7fff640 1 -- 172.21.15.129:0/2486672122 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007cd0 con 0x7fc3f01561e0 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 -- 172.21.15.129:0/2486672122 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 msgr2=0x7fc3f00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 --2- 172.21.15.129:0/2486672122 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f00c8cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009d90 tx=0x7fc3e802a530 comp rx=0 tx=0).stop 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 -- 172.21.15.129:0/2486672122 shutdown_connections 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 --2- 172.21.15.129:0/2486672122 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f00c8cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 --2- 172.21.15.129:0/2486672122 >> v2:172.21.15.171:3300/0 conn(0x7fc3f0152fa0 0x7fc3f00ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 -- 172.21.15.129:0/2486672122 >> 172.21.15.129:0/2486672122 conn(0x7fc3f006c400 msgr2=0x7fc3f006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 -- 172.21.15.129:0/2486672122 shutdown_connections 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.132+0000 7fc3f7609640 1 -- 172.21.15.129:0/2486672122 wait complete. 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.133+0000 7fc3f7609640 1 Processor -- start 2024-04-25T01:27:56.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.133+0000 7fc3f7609640 1 -- start start 2024-04-25T01:27:56.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f7609640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3f00c5520 0x7fc3f00c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f7609640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 0x7fc3f00c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f7609640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f7609640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f00c9200 con 0x7fc3f0152fa0 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f7609640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f00c6030 con 0x7fc3f00c5520 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f7609640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f00c9380 con 0x7fc3f01561e0 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f4ba4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 0x7fc3f00c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f53a5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3f00c5520 0x7fc3f00c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f4ba4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 0x7fc3f00c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56940/0 (socket says 172.21.15.129:56940) 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f4ba4640 1 -- 172.21.15.129:0/3587158851 learned_addr learned my addr 172.21.15.129:0/3587158851 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.134+0000 7fc3f5ba6640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f5ba6640 1 -- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3f00c5520 msgr2=0x7fc3f00c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f5ba6640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3f00c5520 0x7fc3f00c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f5ba6640 1 -- 172.21.15.129:0/3587158851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 msgr2=0x7fc3f00c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f5ba6640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 0x7fc3f00c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f5ba6640 1 -- 172.21.15.129:0/3587158851 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f00c4b60 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f53a5640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3f00c5520 0x7fc3f00c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f4ba4640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 0x7fc3f00c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f5ba6640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f014e5d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8031000 tx=0x7fc3e8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e803f070 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc3e8031bc0 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8044460 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f0158fd0 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.135+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f00c4e00 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.136+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f0075550 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.137+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3e8044600 con 0x7fc3f01561e0 2024-04-25T01:27:56.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.137+0000 7fc3e5ffb640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3d406be30 0x7fc3d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.137+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc3e80ac940 con 0x7fc3f01561e0 2024-04-25T01:27:56.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.137+0000 7fc3f53a5640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3d406be30 0x7fc3d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.138+0000 7fc3f53a5640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3d406be30 0x7fc3d406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009560 tx=0x7fc3ec009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.141+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3e807a800 con 0x7fc3f01561e0 2024-04-25T01:27:56.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.352+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f00ca0c0 con 0x7fc3f01561e0 2024-04-25T01:27:56.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.450+0000 7fc3e5ffb640 1 -- 172.21.15.129:0/3587158851 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fc3e807eb00 con 0x7fc3f01561e0 2024-04-25T01:27:56.429 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:56.430 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:54.802314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:27:56.430 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-25T01:27:56.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3d406be30 msgr2=0x7fc3d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3d406be30 0x7fc3d406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009560 tx=0x7fc3ec009190 comp rx=0 tx=0).stop 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 msgr2=0x7fc3f014e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f014e5d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8031000 tx=0x7fc3e8004a40 comp rx=0 tx=0).stop 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 shutdown_connections 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fc3d406be30 0x7fc3d406e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc3f01561e0 0x7fc3f014e5d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f0152fa0 0x7fc3f00c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 --2- 172.21.15.129:0/3587158851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3f00c5520 0x7fc3f00c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 >> 172.21.15.129:0/3587158851 conn(0x7fc3f006c400 msgr2=0x7fc3f00c7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 shutdown_connections 2024-04-25T01:27:56.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.452+0000 7fc3f7609640 1 -- 172.21.15.129:0/3587158851 wait complete. 2024-04-25T01:27:56.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:27:56.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.562+0000 7fd37d4cd640 1 Processor -- start 2024-04-25T01:27:56.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.562+0000 7fd37d4cd640 1 -- start start 2024-04-25T01:27:56.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd37d4cd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3780ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd37d4cd640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd3781561e0 0x7fd3780c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd37d4cd640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3780c9860 con 0x7fd378152fa0 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd37d4cd640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fd378156690 con 0x7fd3781561e0 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd37d4cd640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd3780c5a70 con 0x7fd3780c5520 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd36ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3780ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd3777fe640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd3781561e0 0x7fd3780c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd36ffff640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3780ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:57788/0 (socket says 172.21.15.129:57788) 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd376ffd640 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fd378152fa0 0x7fd3780c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.129:57670/0 (socket says 172.21.15.129:57670) 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd3777fe640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd3781561e0 0x7fd3780c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54476/0 (socket says 172.21.15.129:54476) 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.563+0000 7fd36ffff640 1 -- 172.21.15.129:0/3827036926 learned_addr learned my addr 172.21.15.129:0/3827036926 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.564+0000 7fd36ffff640 1 -- 172.21.15.129:0/3827036926 >> v2:172.21.15.171:3300/0 conn(0x7fd3781561e0 msgr2=0x7fd3780c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.564+0000 7fd36ffff640 1 --2- 172.21.15.129:0/3827036926 >> v2:172.21.15.171:3300/0 conn(0x7fd3781561e0 0x7fd3780c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.564+0000 7fd36ffff640 1 -- 172.21.15.129:0/3827036926 >> v1:172.21.15.171:6789/0 conn(0x7fd378152fa0 legacy=0x7fd3780c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.564+0000 7fd36ffff640 1 -- 172.21.15.129:0/3827036926 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3780c5d50 con 0x7fd3780c5520 2024-04-25T01:27:56.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.564+0000 7fd36ffff640 1 --2- 172.21.15.129:0/3827036926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3780ca2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd360009870 tx=0x7fd36002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2ff92b3e62c9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.564+0000 7fd3767fc640 1 -- 172.21.15.129:0/3827036926 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360037070 con 0x7fd3780c5520 2024-04-25T01:27:56.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd3767fc640 1 -- 172.21.15.129:0/3827036926 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd360032440 con 0x7fd3780c5520 2024-04-25T01:27:56.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd3767fc640 1 -- 172.21.15.129:0/3827036926 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360032830 con 0x7fd3780c5520 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 -- 172.21.15.129:0/3827036926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 msgr2=0x7fd3780ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/3827036926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3780ca2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd360009870 tx=0x7fd36002a130 comp rx=0 tx=0).stop 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 -- 172.21.15.129:0/3827036926 shutdown_connections 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/3827036926 >> v2:172.21.15.171:3300/0 conn(0x7fd3781561e0 0x7fd3780c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/3827036926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3780ca2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 -- 172.21.15.129:0/3827036926 >> 172.21.15.129:0/3827036926 conn(0x7fd37806c400 msgr2=0x7fd37806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 -- 172.21.15.129:0/3827036926 shutdown_connections 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.565+0000 7fd37d4cd640 1 -- 172.21.15.129:0/3827036926 wait complete. 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.566+0000 7fd37d4cd640 1 Processor -- start 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.566+0000 7fd37d4cd640 1 -- start start 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd37d4cd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3781e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd37d4cd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 0x7fd3780c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd37d4cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3781561e0 0x7fd3781e07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd37d4cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780c9200 con 0x7fd3781561e0 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd37d4cd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd3780c6030 con 0x7fd378152fa0 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd37d4cd640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fd3780c9380 con 0x7fd3780c5520 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd376ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3781e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd3777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3781561e0 0x7fd3781e07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 0x7fd3780c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd3777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3781561e0 0x7fd3781e07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56946/0 (socket says 172.21.15.129:56946) 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd376ffd640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3781e1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.129:57794/0 (socket says 172.21.15.129:57794) 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 0x7fd3780c11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:54492/0 (socket says 172.21.15.129:54492) 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd3777fe640 1 -- 172.21.15.129:0/313905082 learned_addr learned my addr 172.21.15.129:0/313905082 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 -- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 msgr2=0x7fd3781e1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3781e1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 -- 172.21.15.129:0/313905082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3781561e0 msgr2=0x7fd3781e07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3781561e0 0x7fd3781e07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd36ffff640 1 -- 172.21.15.129:0/313905082 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3781e26e0 con 0x7fd378152fa0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.567+0000 7fd376ffd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3781e1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.568+0000 7fd36ffff640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 0x7fd3780c11e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd360031000 tx=0x7fd3600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.568+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360041020 con 0x7fd378152fa0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.568+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd3600306c0 con 0x7fd378152fa0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.568+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3781ec410 con 0x7fd378152fa0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.568+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360037040 con 0x7fd378152fa0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.568+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3780c1720 con 0x7fd378152fa0 2024-04-25T01:27:56.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.569+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd378075550 con 0x7fd378152fa0 2024-04-25T01:27:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.569+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd360044020 con 0x7fd378152fa0 2024-04-25T01:27:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.570+0000 7fd36f7fe640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35806be30 0x7fd35806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.570+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd3600aba00 con 0x7fd378152fa0 2024-04-25T01:27:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.570+0000 7fd376ffd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35806be30 0x7fd35806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.570+0000 7fd376ffd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35806be30 0x7fd35806e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd368007fd0 tx=0x7fd368007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.578+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd36004b030 con 0x7fd378152fa0 2024-04-25T01:27:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.797+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3780ca0c0 con 0x7fd378152fa0 2024-04-25T01:27:56.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.798+0000 7fd36f7fe640 1 -- 172.21.15.129:0/313905082 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fd360079830 con 0x7fd378152fa0 2024-04-25T01:27:56.777 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:56.777 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6834/3271880398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3271880398},{"type":"v1","addr":"172.21.15.129:6834","nonce":3271880398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/803568812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6833","nonce":803568812},{"type":"v1","addr":"172.21.15.129:6835","nonce":803568812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/857370742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":857370742},{"type":"v1","addr":"172.21.15.171:6836","nonce":857370742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:01:32.731107+0000","modified":"2024-04-25T01:01:54.802314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/1355919064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":1355919064},{"type":"v1","addr":"172.21.15.171:6837","nonce":1355919064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-04-25T01:27:56.777 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35806be30 msgr2=0x7fd35806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35806be30 0x7fd35806e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd368007fd0 tx=0x7fd368007e80 comp rx=0 tx=0).stop 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 msgr2=0x7fd3780c11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 0x7fd3780c11e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd360031000 tx=0x7fd3600046c0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 shutdown_connections 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fd35806be30 0x7fd35806e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3781561e0 0x7fd3781e07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd378152fa0 0x7fd3780c11e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 --2- 172.21.15.129:0/313905082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd3780c5520 0x7fd3781e1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 >> 172.21.15.129:0/313905082 conn(0x7fd37806c400 msgr2=0x7fd3780ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:56.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 shutdown_connections 2024-04-25T01:27:56.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.800+0000 7fd37d4cd640 1 -- 172.21.15.129:0/313905082 wait complete. 2024-04-25T01:27:56.789 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:27:56.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.908+0000 7f580b7d5640 1 Processor -- start 2024-04-25T01:27:56.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.908+0000 7f580b7d5640 1 -- start start 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f580b7d5640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5804152fa0 0x7f58040ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f580b7d5640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f58040c5520 0x7f58041561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f580b7d5640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58040c9860 con 0x7f58040c7e50 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f580b7d5640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f5804156720 con 0x7f5804152fa0 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f580b7d5640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f58040c5a70 con 0x7f58040c5520 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f5809571640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5804152fa0 0x7f58040ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f5809571640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f5804152fa0 0x7f58040ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56952/0 (socket says 172.21.15.129:56952) 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f5809571640 1 -- 172.21.15.129:0/3616503229 learned_addr learned my addr 172.21.15.129:0/3616503229 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.909+0000 7f5808d70640 1 --2- 172.21.15.129:0/3616503229 >> v2:172.21.15.171:3300/0 conn(0x7f58040c5520 0x7f58041561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.910+0000 7f5809571640 1 -- 172.21.15.129:0/3616503229 >> v2:172.21.15.171:3300/0 conn(0x7f58040c5520 msgr2=0x7f58041561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.910+0000 7f5809571640 1 --2- 172.21.15.129:0/3616503229 >> v2:172.21.15.171:3300/0 conn(0x7f58040c5520 0x7f58041561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.910+0000 7f5809571640 1 -- 172.21.15.129:0/3616503229 >> v1:172.21.15.171:6789/0 conn(0x7f58040c7e50 legacy=0x7f58040c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.910+0000 7f5809571640 1 -- 172.21.15.129:0/3616503229 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58040c5d50 con 0x7f5804152fa0 2024-04-25T01:27:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.910+0000 7f5809571640 1 --2- 172.21.15.129:0/3616503229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804152fa0 0x7f58040ca2e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f580408c330 tx=0x7f57f402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe78c40cdb69bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.910+0000 7f57fbfff640 1 -- 172.21.15.129:0/3616503229 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4037070 con 0x7f5804152fa0 2024-04-25T01:27:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f57fbfff640 1 -- 172.21.15.129:0/3616503229 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f57f4032440 con 0x7f5804152fa0 2024-04-25T01:27:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f57fbfff640 1 -- 172.21.15.129:0/3616503229 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4032830 con 0x7f5804152fa0 2024-04-25T01:27:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 -- 172.21.15.129:0/3616503229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804152fa0 msgr2=0x7f58040ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.893 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 --2- 172.21.15.129:0/3616503229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804152fa0 0x7f58040ca2e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f580408c330 tx=0x7f57f402a150 comp rx=0 tx=0).stop 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 -- 172.21.15.129:0/3616503229 shutdown_connections 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 --2- 172.21.15.129:0/3616503229 >> v2:172.21.15.171:3300/0 conn(0x7f58040c5520 0x7f58041561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 --2- 172.21.15.129:0/3616503229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5804152fa0 0x7f58040ca2e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 -- 172.21.15.129:0/3616503229 >> 172.21.15.129:0/3616503229 conn(0x7f580406c400 msgr2=0x7f580406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 -- 172.21.15.129:0/3616503229 shutdown_connections 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.911+0000 7f580b7d5640 1 -- 172.21.15.129:0/3616503229 wait complete. 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.912+0000 7f580b7d5640 1 Processor -- start 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.912+0000 7f580b7d5640 1 -- start start 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f580b7d5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58040c5520 0x7f58041ef170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f580b7d5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 0x7f58040c0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f580b7d5640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5804152fa0 0x7f58040c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f580b7d5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58040c93d0 con 0x7f58040c7e50 2024-04-25T01:27:56.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f580b7d5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f58040c6030 con 0x7f58040c5520 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f580b7d5640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f58040c9550 con 0x7f5804152fa0 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 0x7f58040c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 0x7f58040c0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56956/0 (socket says 172.21.15.129:56956) 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 -- 172.21.15.129:0/1735634682 learned_addr learned my addr 172.21.15.129:0/1735634682 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5809571640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58040c5520 0x7f58041ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5809d72640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5804152fa0 0x7f58040c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 -- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5804152fa0 msgr2=0x7f58040c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5804152fa0 0x7f58040c06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 -- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58040c5520 msgr2=0x7f58041ef170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58040c5520 0x7f58041ef170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 -- 172.21.15.129:0/1735634682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58040c4e30 con 0x7f58040c7e50 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.913+0000 7f5808d70640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 0x7f58040c0170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f580000e9b0 tx=0x7f580000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.914+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800017070 con 0x7f58040c7e50 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.914+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f580000c9a0 con 0x7f58040c7e50 2024-04-25T01:27:56.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.914+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580000cdb0 con 0x7f58040c7e50 2024-04-25T01:27:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.914+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58041ece00 con 0x7f58040c7e50 2024-04-25T01:27:56.896 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.914+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58040c4870 con 0x7f58040c7e50 2024-04-25T01:27:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.915+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5804075550 con 0x7f58040c7e50 2024-04-25T01:27:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.916+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f580001a070 con 0x7f58040c7e50 2024-04-25T01:27:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.916+0000 7f57f9ffb640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57e806be30 0x7f57e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.916+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f580008b150 con 0x7f58040c7e50 2024-04-25T01:27:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.916+0000 7f5809571640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57e806be30 0x7f57e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:56.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.917+0000 7f5809571640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57e806be30 0x7f57e806e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57f4034000 tx=0x7f57f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:56.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:56.921+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5800059140 con 0x7f58040c7e50 2024-04-25T01:27:57.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.102+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58040cb930 con 0x7f58040c7e50 2024-04-25T01:27:57.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.103+0000 7f57f9ffb640 1 -- 172.21.15.129:0/1735634682 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18891 (secure 0 0 0) 0x7f580005d440 con 0x7f58040c7e50 2024-04-25T01:27:57.082 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-25T01:27:57.083 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"fsid":"4b43d0d1-0ff7-4061-ba99-dacfcc159ca4","created":"2024-04-25T01:01:08.472149+0000","modified":"2024-04-25T01:02:13.734125+0000","last_up_change":"2024-04-25T01:01:19.317187+0000","last_in_change":"2024-04-25T01:01:10.595760+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:01:20.693388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:01:30.640376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:01:31.706336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:01:35.424628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d671da37-185a-43f0-b212-f63d250d2c5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6801","nonce":1195619493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6803","nonce":1195619493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6807","nonce":1195619493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":1195619493},{"type":"v1","addr":"172.21.15.129:6805","nonce":1195619493}]},"public_addr":"172.21.15.129:6801/1195619493","cluster_addr":"172.21.15.129:6803/1195619493","heartbeat_back_addr":"172.21.15.129:6807/1195619493","heartbeat_front_addr":"172.21.15.129:6805/1195619493","state":["exists","up"]},{"osd":1,"uuid":"c03a61c0-f685-4a43-acec-4edbe0dfd835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6817","nonce":3197437953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6819","nonce":3197437953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6823","nonce":3197437953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3197437953},{"type":"v1","addr":"172.21.15.129:6821","nonce":3197437953}]},"public_addr":"172.21.15.129:6817/3197437953","cluster_addr":"172.21.15.129:6819/3197437953","heartbeat_back_addr":"172.21.15.129:6823/3197437953","heartbeat_front_addr":"172.21.15.129:6821/3197437953","state":["exists","up"]},{"osd":2,"uuid":"8e4dfd32-d051-444e-8789-a462191b5440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6825","nonce":1981636327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6827","nonce":1981636327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6831","nonce":1981636327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1981636327},{"type":"v1","addr":"172.21.15.129:6829","nonce":1981636327}]},"public_addr":"172.21.15.129:6825/1981636327","cluster_addr":"172.21.15.129:6827/1981636327","heartbeat_back_addr":"172.21.15.129:6831/1981636327","heartbeat_front_addr":"172.21.15.129:6829/1981636327","state":["exists","up"]},{"osd":3,"uuid":"582dca5d-083d-4052-8146-9d3f5dce56bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6809","nonce":4291613316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6811","nonce":4291613316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6815","nonce":4291613316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":4291613316},{"type":"v1","addr":"172.21.15.129:6813","nonce":4291613316}]},"public_addr":"172.21.15.129:6809/4291613316","cluster_addr":"172.21.15.129:6811/4291613316","heartbeat_back_addr":"172.21.15.129:6815/4291613316","heartbeat_front_addr":"172.21.15.129:6813/4291613316","state":["exists","up"]},{"osd":4,"uuid":"5bfc9318-9d58-456c-81de-d3e7d2231fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6810","nonce":4119213215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6814","nonce":4119213215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6822","nonce":4119213215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4119213215},{"type":"v1","addr":"172.21.15.171:6818","nonce":4119213215}]},"public_addr":"172.21.15.171:6810/4119213215","cluster_addr":"172.21.15.171:6814/4119213215","heartbeat_back_addr":"172.21.15.171:6822/4119213215","heartbeat_front_addr":"172.21.15.171:6818/4119213215","state":["exists","up"]},{"osd":5,"uuid":"62b60dcc-6111-4df1-a716-f73967a074a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6809","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6811","nonce":486235959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6813","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6815","nonce":486235959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6823","nonce":486235959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":486235959},{"type":"v1","addr":"172.21.15.171:6819","nonce":486235959}]},"public_addr":"172.21.15.171:6811/486235959","cluster_addr":"172.21.15.171:6815/486235959","heartbeat_back_addr":"172.21.15.171:6823/486235959","heartbeat_front_addr":"172.21.15.171:6819/486235959","state":["exists","up"]},{"osd":6,"uuid":"f0bd17a6-a68f-4161-8a2c-c9ba62c68e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6825","nonce":2873795804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6827","nonce":2873795804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6831","nonce":2873795804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2873795804},{"type":"v1","addr":"172.21.15.171:6829","nonce":2873795804}]},"public_addr":"172.21.15.171:6825/2873795804","cluster_addr":"172.21.15.171:6827/2873795804","heartbeat_back_addr":"172.21.15.171:6831/2873795804","heartbeat_front_addr":"172.21.15.171:6829/2873795804","state":["exists","up"]},{"osd":7,"uuid":"93678b87-0929-4133-ad43-1f15219e89e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6801","nonce":3134257063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6803","nonce":3134257063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6807","nonce":3134257063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3134257063},{"type":"v1","addr":"172.21.15.171:6805","nonce":3134257063}]},"public_addr":"172.21.15.171:6801/3134257063","cluster_addr":"172.21.15.171:6803/3134257063","heartbeat_back_addr":"172.21.15.171:6807/3134257063","heartbeat_front_addr":"172.21.15.171:6805/3134257063","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.781118+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.784857+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.795641+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.705300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.798852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.759871+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.817147+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:01:15.727490+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":1}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:27:57.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57e806be30 msgr2=0x7f57e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57e806be30 0x7f57e806e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57f4034000 tx=0x7f57f402e000 comp rx=0 tx=0).stop 2024-04-25T01:27:57.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 msgr2=0x7f58040c0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 0x7f58040c0170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f580000e9b0 tx=0x7f580000ee70 comp rx=0 tx=0).stop 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 shutdown_connections 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f57e806be30 0x7f57e806e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5804152fa0 0x7f58040c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58040c7e50 0x7f58040c0170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 --2- 172.21.15.129:0/1735634682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58040c5520 0x7f58041ef170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 >> 172.21.15.129:0/1735634682 conn(0x7f580406c400 msgr2=0x7f580406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 shutdown_connections 2024-04-25T01:27:57.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.105+0000 7f580b7d5640 1 -- 172.21.15.129:0/1735634682 wait complete. 2024-04-25T01:27:57.095 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T01:27:57.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.216+0000 7f1cf2742640 1 Processor -- start 2024-04-25T01:27:57.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.216+0000 7f1cf2742640 1 -- start start 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.216+0000 7f1cf2742640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cec1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.216+0000 7f1cf2742640 1 -- --> v1:172.21.15.129:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cec0c9860 con 0x7f1cec152fa0 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.216+0000 7f1cf2742640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cec0c5a70 con 0x7f1cec0c5520 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.216+0000 7f1cf2742640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1cec156720 con 0x7f1cec0ca2e0 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1cebfff640 1 --1- >> v1:172.21.15.129:6789/0 conn(0x7f1cec152fa0 0x7f1cec0c7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.129:6789/0 says I am v1:172.21.15.129:53516/0 (socket says 172.21.15.129:53516) 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1cebfff640 1 -- 172.21.15.129:0/2626326012 learned_addr learned my addr 172.21.15.129:0/2626326012 (peer_addr_for_me v1:172.21.15.129:0/0) 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cec1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.0 v1:172.21.15.129:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560374886 0 0) 0x7f1cdc001110 con 0x7f1cec152fa0 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 --> v1:172.21.15.129:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cd4003520 con 0x7f1cec152fa0 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146517272 0 0) 0x7f1cec0c5a70 con 0x7f1cec0c5520 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cdc001110 con 0x7f1cec0c5520 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.0 v1:172.21.15.129:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1347091133 0 0) 0x7f1cd4003520 con 0x7f1cec152fa0 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 msgr2=0x7f1cec1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cec1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 >> v1:172.21.15.171:6789/0 conn(0x7f1cec0c5520 legacy=0x7f1cec0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 --> v1:172.21.15.129:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec0c5d50 con 0x7f1cec152fa0 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.0 v1:172.21.15.129:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 794772415 0 0) 0x7f1cdc001840 con 0x7f1cec152fa0 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cec1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:57.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 >> v1:172.21.15.129:6789/0 conn(0x7f1cec152fa0 legacy=0x7f1cec0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cd4003b70 0x7f1cd4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec156720 con 0x7f1cd4004d40 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd400bb50 con 0x7f1cd4003b70 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd400bd50 con 0x7f1cec0ca2e0 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.217+0000 7f1cebfff640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cd4003b70 0x7f1cd4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1cf0cdf640 1 -- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 msgr2=0x7f1cd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1cf0cdf640 1 -- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cd4003b70 msgr2=0x7f1cd4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cd4003b70 0x7f1cd4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1cf0cdf640 1 -- 172.21.15.129:0/2626326012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd4003520 con 0x7f1cd4004d40 2024-04-25T01:27:57.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec0ca2e0 0x7f1cd4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e350 tx=0x7f1cd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf280830bdf908e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.218+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4033070 con 0x7f1cd4004d40 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1ce402f660 con 0x7f1cd4004d40 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1ceaffd640 1 -- 172.21.15.129:0/2626326012 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4009de0 con 0x7f1cd4004d40 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf2742640 1 -- 172.21.15.129:0/2626326012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd4004d40 msgr2=0x7f1cd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf2742640 1 --2- 172.21.15.129:0/2626326012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e350 tx=0x7f1cd4005830 comp rx=0 tx=0).stop 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf0cdf640 1 -- 172.21.15.129:0/2626326012 reap_dead start 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf2742640 1 -- 172.21.15.129:0/2626326012 shutdown_connections 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf2742640 1 -- 172.21.15.129:0/2626326012 >> 172.21.15.129:0/2626326012 conn(0x7f1cec06c400 msgr2=0x7f1cec06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf2742640 1 -- 172.21.15.129:0/2626326012 shutdown_connections 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.219+0000 7f1cf2742640 1 -- 172.21.15.129:0/2626326012 wait complete. 2024-04-25T01:27:57.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.220+0000 7f1cf2742640 1 Processor -- start 2024-04-25T01:27:57.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.220+0000 7f1cf2742640 1 -- start start 2024-04-25T01:27:57.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf2742640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec157de0 0x7f1cec1eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf2742640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 0x7f1cec1eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf2742640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec1ef6a0 0x7f1cec0c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf2742640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec0c6030 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf2742640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec0ca770 con 0x7f1cec1ef6a0 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf2742640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f1cec0c93d0 con 0x7f1cec157de0 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 0x7f1cec1eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 0x7f1cec1eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56982/0 (socket says 172.21.15.129:56982) 2024-04-25T01:27:57.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/4220773958 learned_addr learned my addr 172.21.15.129:0/4220773958 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cebfff640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec157de0 0x7f1cec1eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1cf0cdf640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec1ef6a0 0x7f1cec0c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec157de0 msgr2=0x7f1cec1eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec157de0 0x7f1cec1eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec1ef6a0 msgr2=0x7f1cec0c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec1ef6a0 0x7f1cec0c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.221+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/4220773958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec0bf140 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.222+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 0x7f1cec1eb9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000f8e0 tx=0x7f1ce000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.222+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00053a0 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.222+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1ce0018070 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.222+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000d910 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.222+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec0bf420 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.222+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1eeea0 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.223+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec075550 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.223+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ce000dab0 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.223+0000 7f1ce8ff9640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1cd806be30 0x7f1cd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.224+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1ce008b320 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.224+0000 7f1cebfff640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1cd806be30 0x7f1cd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.224+0000 7f1cebfff640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1cd806be30 0x7f1cd806e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0010e0 tx=0x7f1cdc007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:57.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.228+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ce0059150 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.445+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1cec0c4fd0 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.447+0000 7f1ce8ff9640 1 -- 172.21.15.129:0/4220773958 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f1ce008b0e0 con 0x7f1cec0c4aa0 2024-04-25T01:27:57.426 INFO:teuthology.orchestra.run.smithi129.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:27:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1cd806be30 msgr2=0x7f1cd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1cd806be30 0x7f1cd806e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0010e0 tx=0x7f1cdc007a20 comp rx=0 tx=0).stop 2024-04-25T01:27:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 msgr2=0x7f1cec1eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 0x7f1cec1eb9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000f8e0 tx=0x7f1ce000fda0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 shutdown_connections 2024-04-25T01:27:57.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f1cd806be30 0x7f1cd806e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec1ef6a0 0x7f1cec0c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec0c4aa0 0x7f1cec1eb9f0 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000f8e0 tx=0x7f1ce000fda0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 --2- 172.21.15.129:0/4220773958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cec157de0 0x7f1cec1eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 >> 172.21.15.129:0/4220773958 conn(0x7f1cec06c400 msgr2=0x7f1cec06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:57.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 shutdown_connections 2024-04-25T01:27:57.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.451+0000 7f1cf2742640 1 -- 172.21.15.129:0/4220773958 wait complete. 2024-04-25T01:27:57.441 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T01:27:57.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 Processor -- start 2024-04-25T01:27:57.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 -- start start 2024-04-25T01:27:57.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7efc201528f0 0x7efc2006c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7efc200c74a0 0x7efc200c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc2006cd80 con 0x7efc20156100 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7efc200c7e60 con 0x7efc201528f0 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.564+0000 7efc275e7640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7efc2006bd60 con 0x7efc200c74a0 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7efc201528f0 0x7efc2006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7efc201528f0 0x7efc2006c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38854/0 (socket says 172.21.15.129:38854) 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 -- 172.21.15.129:0/2293478927 learned_addr learned my addr 172.21.15.129:0/2293478927 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25b84640 1 --2- 172.21.15.129:0/2293478927 >> v2:172.21.15.171:3300/0 conn(0x7efc200c74a0 0x7efc200c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 -- 172.21.15.129:0/2293478927 >> v2:172.21.15.171:3300/0 conn(0x7efc200c74a0 msgr2=0x7efc200c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 --2- 172.21.15.129:0/2293478927 >> v2:172.21.15.171:3300/0 conn(0x7efc200c74a0 0x7efc200c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 -- 172.21.15.129:0/2293478927 >> v1:172.21.15.171:6789/0 conn(0x7efc20156100 legacy=0x7efc201568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25383640 1 -- 172.21.15.129:0/2293478927 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2006c040 con 0x7efc201528f0 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.565+0000 7efc25b84640 1 --2- 172.21.15.129:0/2293478927 >> v2:172.21.15.171:3300/0 conn(0x7efc200c74a0 0x7efc200c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:27:57.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc25383640 1 --2- 172.21.15.129:0/2293478927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc2006c840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc20074f20 tx=0x7efc1c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b552ada138c29c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:57.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc17fff640 1 -- 172.21.15.129:0/2293478927 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c036070 con 0x7efc201528f0 2024-04-25T01:27:57.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc17fff640 1 -- 172.21.15.129:0/2293478927 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efc1c031600 con 0x7efc201528f0 2024-04-25T01:27:57.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc17fff640 1 -- 172.21.15.129:0/2293478927 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0319f0 con 0x7efc201528f0 2024-04-25T01:27:57.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 -- 172.21.15.129:0/2293478927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 msgr2=0x7efc2006c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 --2- 172.21.15.129:0/2293478927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc2006c840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc20074f20 tx=0x7efc1c02a730 comp rx=0 tx=0).stop 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 -- 172.21.15.129:0/2293478927 shutdown_connections 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 --2- 172.21.15.129:0/2293478927 >> v2:172.21.15.171:3300/0 conn(0x7efc200c74a0 0x7efc200c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 --2- 172.21.15.129:0/2293478927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc2006c840 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 -- 172.21.15.129:0/2293478927 >> 172.21.15.129:0/2293478927 conn(0x7efc200ac9a0 msgr2=0x7efc200acdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 -- 172.21.15.129:0/2293478927 shutdown_connections 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.566+0000 7efc275e7640 1 -- 172.21.15.129:0/2293478927 wait complete. 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.567+0000 7efc275e7640 1 Processor -- start 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.567+0000 7efc275e7640 1 -- start start 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc275e7640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efc200c74a0 0x7efc200cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc275e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc201ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc275e7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc20156100 0x7efc200ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc275e7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc20152e50 con 0x7efc201528f0 2024-04-25T01:27:57.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc275e7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efc20152cd0 con 0x7efc20156100 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc275e7640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7efc200c5ad0 con 0x7efc200c74a0 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc24b82640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc201ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc24b82640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc201ebf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38870/0 (socket says 172.21.15.129:38870) 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc24b82640 1 -- 172.21.15.129:0/583461749 learned_addr learned my addr 172.21.15.129:0/583461749 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc25b84640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc20156100 0x7efc200ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc25383640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efc200c74a0 0x7efc200cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc24b82640 1 -- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efc200c74a0 msgr2=0x7efc200cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.568+0000 7efc24b82640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efc200c74a0 0x7efc200cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc24b82640 1 -- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc20156100 msgr2=0x7efc200ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc24b82640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc20156100 0x7efc200ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc24b82640 1 -- 172.21.15.129:0/583461749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2014bed0 con 0x7efc201528f0 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc24b82640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc201ebf00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c440 tx=0x7efc1000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:57.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc100052d0 con 0x7efc201528f0 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efc10005470 con 0x7efc201528f0 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc200caed0 con 0x7efc201528f0 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10007280 con 0x7efc201528f0 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.569+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc200c4720 con 0x7efc201528f0 2024-04-25T01:27:57.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.570+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc20075550 con 0x7efc201528f0 2024-04-25T01:27:57.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.571+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc10007420 con 0x7efc201528f0 2024-04-25T01:27:57.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.571+0000 7efc15ffb640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7efc0406be30 0x7efc0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:57.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.572+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7efc1008a6c0 con 0x7efc201528f0 2024-04-25T01:27:57.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.572+0000 7efc25383640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7efc0406be30 0x7efc0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:57.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.572+0000 7efc25383640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7efc0406be30 0x7efc0406e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc1c033000 tx=0x7efc1c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:57.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.576+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc100585c0 con 0x7efc201528f0 2024-04-25T01:27:57.726 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-04-25T01:27:57.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:57.797+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efc20068630 con 0x7efc201528f0 2024-04-25T01:27:58.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.448+0000 7efc15ffb640 1 -- 172.21.15.129:0/583461749 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7efc1005c8c0 con 0x7efc201528f0 2024-04-25T01:27:58.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7efc0406be30 msgr2=0x7efc0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7efc0406be30 0x7efc0406e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc1c033000 tx=0x7efc1c009000 comp rx=0 tx=0).stop 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 msgr2=0x7efc201ebf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc201ebf00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c440 tx=0x7efc1000c900 comp rx=0 tx=0).stop 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 shutdown_connections 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7efc0406be30 0x7efc0406e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc20156100 0x7efc200ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc201528f0 0x7efc201ebf00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 --2- 172.21.15.129:0/583461749 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efc200c74a0 0x7efc200cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.450+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 >> 172.21.15.129:0/583461749 conn(0x7efc200ac9a0 msgr2=0x7efc200acd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.451+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 shutdown_connections 2024-04-25T01:27:58.430 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.451+0000 7efc275e7640 1 -- 172.21.15.129:0/583461749 wait complete. 2024-04-25T01:27:58.440 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T01:27:58.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.562+0000 7f521ed22640 1 Processor -- start 2024-04-25T01:27:58.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.562+0000 7f521ed22640 1 -- start start 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f521ed22640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f52180c9030 0x7f5218158970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f521ed22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f5218158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f521ed22640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f521806cfe0 con 0x7f5218152bb0 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f521ed22640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7f52181593f0 con 0x7f52180c9030 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f521ed22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f5218155dc0 con 0x7f52180c8b20 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f5217fff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f52180c9030 0x7f5218158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f521d2bf640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f5218158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f5217fff640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f52180c9030 0x7f5218158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.129:37456/0 (socket says 172.21.15.129:37456) 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.563+0000 7f5217fff640 1 -- 172.21.15.129:0/3488162107 learned_addr learned my addr 172.21.15.129:0/3488162107 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.564+0000 7f5217fff640 1 -- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 msgr2=0x7f5218158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.564+0000 7f5217fff640 1 --2- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f5218158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.564+0000 7f5217fff640 1 -- 172.21.15.129:0/3488162107 >> v1:172.21.15.171:6789/0 conn(0x7f5218152bb0 legacy=0x7f52180c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.564+0000 7f5217fff640 1 -- 172.21.15.129:0/3488162107 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218155490 con 0x7f52180c9030 2024-04-25T01:27:58.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.564+0000 7f521d2bf640 1 --2- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f5218158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.564+0000 7f5217fff640 1 --2- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f5218158970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5204009870 tx=0x7f520402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b98583d4f5eed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f52177fe640 1 -- 172.21.15.129:0/3488162107 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204037070 con 0x7f52180c9030 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f52177fe640 1 -- 172.21.15.129:0/3488162107 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5204032440 con 0x7f52180c9030 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f52177fe640 1 -- 172.21.15.129:0/3488162107 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204032830 con 0x7f52180c9030 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 -- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 msgr2=0x7f5218158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 --2- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f5218158970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5204009870 tx=0x7f520402a1c0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 -- 172.21.15.129:0/3488162107 shutdown_connections 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 --2- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f5218158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 --2- 172.21.15.129:0/3488162107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f5218158970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 -- 172.21.15.129:0/3488162107 >> 172.21.15.129:0/3488162107 conn(0x7f52180acbf0 msgr2=0x7f52180ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 -- 172.21.15.129:0/3488162107 shutdown_connections 2024-04-25T01:27:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.565+0000 7f521ed22640 1 -- 172.21.15.129:0/3488162107 wait complete. 2024-04-25T01:27:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.566+0000 7f521ed22640 1 Processor -- start 2024-04-25T01:27:58.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.566+0000 7f521ed22640 1 -- start start 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521ed22640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f52180adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521ed22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f52180ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521ed22640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 0x7f521814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521ed22640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5218156250 con 0x7f5218152bb0 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521ed22640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5218155a50 con 0x7f52180c9030 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521ed22640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f52180c94f0 con 0x7f52180c8b20 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521d2bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 0x7f521814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521d2bf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 0x7f521814b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38876/0 (socket says 172.21.15.129:38876) 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f5217fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f52180ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521d2bf640 1 -- 172.21.15.129:0/321621029 learned_addr learned my addr 172.21.15.129:0/321621029 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.567+0000 7f521cabe640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f52180adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521d2bf640 1 -- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 msgr2=0x7f52180adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521d2bf640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f52180adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521d2bf640 1 -- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 msgr2=0x7f52180ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521d2bf640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f52180ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521d2bf640 1 -- 172.21.15.129:0/321621029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218158470 con 0x7f5218152bb0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f5217fff640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f52180ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521cabe640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f52180adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521d2bf640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 0x7f521814b9f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f521000efc0 tx=0x7f5210009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210018030 con 0x7f5218152bb0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f521000cde0 con 0x7f5218152bb0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52181ebdc0 con 0x7f5218152bb0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210003bd0 con 0x7f5218152bb0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.568+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181ec060 con 0x7f5218152bb0 2024-04-25T01:27:58.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.569+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5218068fd0 con 0x7f5218152bb0 2024-04-25T01:27:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.570+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5210005d10 con 0x7f5218152bb0 2024-04-25T01:27:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.570+0000 7f52157fa640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51fc06be30 0x7f51fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.570+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f521008bc20 con 0x7f5218152bb0 2024-04-25T01:27:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.570+0000 7f521cabe640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51fc06be30 0x7f51fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.571+0000 7f521cabe640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51fc06be30 0x7f51fc06e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f521806cf20 tx=0x7f520c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:58.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.578+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5210022070 con 0x7f5218152bb0 2024-04-25T01:27:58.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:58.757+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f52180b09d0 con 0x7f5218152bb0 2024-04-25T01:27:59.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.453+0000 7f52157fa640 1 -- 172.21.15.129:0/321621029 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f5210010040 con 0x7f5218152bb0 2024-04-25T01:27:59.432 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' removed 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51fc06be30 msgr2=0x7f51fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51fc06be30 0x7f51fc06e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f521806cf20 tx=0x7f520c009040 comp rx=0 tx=0).stop 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 msgr2=0x7f521814b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 0x7f521814b9f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f521000efc0 tx=0x7f5210009f90 comp rx=0 tx=0).stop 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 shutdown_connections 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f51fc06be30 0x7f51fc06e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5218152bb0 0x7f521814b9f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52180c9030 0x7f52180ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 --2- 172.21.15.129:0/321621029 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52180c8b20 0x7f52180adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 >> 172.21.15.129:0/321621029 conn(0x7f52180acbf0 msgr2=0x7f52181547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 shutdown_connections 2024-04-25T01:27:59.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.456+0000 7f521ed22640 1 -- 172.21.15.129:0/321621029 wait complete. 2024-04-25T01:27:59.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T01:27:59.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.576+0000 7fe4ca5fb640 1 Processor -- start 2024-04-25T01:27:59.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.576+0000 7fe4ca5fb640 1 -- start start 2024-04-25T01:27:59.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4ca5fb640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe4c4152bb0 0x7fe4c40c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4ca5fb640 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe4c40c9030 0x7fe4c4158970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4ca5fb640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c406cfe0 con 0x7fe4c40c8b20 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4ca5fb640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c4155dc0 con 0x7fe4c4152bb0 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4ca5fb640 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c4155490 con 0x7fe4c40c9030 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe4c4152bb0 0x7fe4c40c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7fe4c4152bb0 0x7fe4c40c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38892/0 (socket says 172.21.15.129:38892) 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 -- 172.21.15.129:0/1308866240 learned_addr learned my addr 172.21.15.129:0/1308866240 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4bbfff640 1 --2- 172.21.15.129:0/1308866240 >> v2:172.21.15.171:3300/0 conn(0x7fe4c40c9030 0x7fe4c4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 -- 172.21.15.129:0/1308866240 >> v2:172.21.15.171:3300/0 conn(0x7fe4c40c9030 msgr2=0x7fe4c4158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 --2- 172.21.15.129:0/1308866240 >> v2:172.21.15.171:3300/0 conn(0x7fe4c40c9030 0x7fe4c4158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 -- 172.21.15.129:0/1308866240 >> v1:172.21.15.171:6789/0 conn(0x7fe4c40c8b20 legacy=0x7fe4c406ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.577+0000 7fe4c3fff640 1 -- 172.21.15.129:0/1308866240 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c4155770 con 0x7fe4c4152bb0 2024-04-25T01:27:59.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.578+0000 7fe4c3fff640 1 --2- 172.21.15.129:0/1308866240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c4152bb0 0x7fe4c40c55a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40c7bf0 tx=0x7fe4b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d87bff2e0782b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.578+0000 7fe4c37fe640 1 -- 172.21.15.129:0/1308866240 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4037070 con 0x7fe4c4152bb0 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.578+0000 7fe4c37fe640 1 -- 172.21.15.129:0/1308866240 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe4b4032440 con 0x7fe4c4152bb0 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.578+0000 7fe4c37fe640 1 -- 172.21.15.129:0/1308866240 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4032830 con 0x7fe4c4152bb0 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/1308866240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c4152bb0 msgr2=0x7fe4c40c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/1308866240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c4152bb0 0x7fe4c40c55a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40c7bf0 tx=0x7fe4b402a1e0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/1308866240 shutdown_connections 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/1308866240 >> v2:172.21.15.171:3300/0 conn(0x7fe4c40c9030 0x7fe4c4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/1308866240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c4152bb0 0x7fe4c40c55a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/1308866240 >> 172.21.15.129:0/1308866240 conn(0x7fe4c40acbf0 msgr2=0x7fe4c40ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/1308866240 shutdown_connections 2024-04-25T01:27:59.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.579+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/1308866240 wait complete. 2024-04-25T01:27:59.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.580+0000 7fe4ca5fb640 1 Processor -- start 2024-04-25T01:27:59.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.580+0000 7fe4ca5fb640 1 -- start start 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe4c40c8b20 0x7fe4c414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 0x7fe4c41eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4c4152bb0 0x7fe4c41eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c40c94f0 con 0x7fe4c40c9030 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4156250 con 0x7fe4c4152bb0 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c406d3d0 con 0x7fe4c40c8b20 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 0x7fe4c41eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 0x7fe4c41eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38900/0 (socket says 172.21.15.129:38900) 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 -- 172.21.15.129:0/3069502458 learned_addr learned my addr 172.21.15.129:0/3069502458 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4c3fff640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe4c40c8b20 0x7fe4c414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4c8b98640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4c4152bb0 0x7fe4c41eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 -- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe4c40c8b20 msgr2=0x7fe4c414b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe4c40c8b20 0x7fe4c414b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 -- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4c4152bb0 msgr2=0x7fe4c41eb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4c4152bb0 0x7fe4c41eb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 -- 172.21.15.129:0/3069502458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41ebb20 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4bbfff640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 0x7fe4c41eb0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e9b0 tx=0x7fe4ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac017070 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe4ac00c9a0 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00cdb0 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c41ef470 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.581+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41ee990 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.582+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c4068fd0 con 0x7fe4c40c9030 2024-04-25T01:27:59.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.583+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4ac01a070 con 0x7fe4c40c9030 2024-04-25T01:27:59.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.583+0000 7fe4c17fa640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe4a406be30 0x7fe4a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:59.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.584+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7fe4ac08b080 con 0x7fe4c40c9030 2024-04-25T01:27:59.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.584+0000 7fe4c3fff640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe4a406be30 0x7fe4a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:59.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.584+0000 7fe4c3fff640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe4a406be30 0x7fe4a406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4034000 tx=0x7fe4b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:59.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.588+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe4ac059140 con 0x7fe4c40c9030 2024-04-25T01:27:59.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:27:59.767+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe4c406d1b0 con 0x7fe4c40c9030 2024-04-25T01:28:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.457+0000 7fe4c17fa640 1 -- 172.21.15.129:0/3069502458 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fe4ac05d440 con 0x7fe4c40c9030 2024-04-25T01:28:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' removed 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe4a406be30 msgr2=0x7fe4a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe4a406be30 0x7fe4a406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4034000 tx=0x7fe4b402e000 comp rx=0 tx=0).stop 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 msgr2=0x7fe4c41eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 0x7fe4c41eb0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e9b0 tx=0x7fe4ac00ee70 comp rx=0 tx=0).stop 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 shutdown_connections 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7fe4a406be30 0x7fe4a406e2f0 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4034000 tx=0x7fe4b402e000 comp rx=0 tx=0).stop 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4c4152bb0 0x7fe4c41eb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c40c9030 0x7fe4c41eb0a0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e9b0 tx=0x7fe4ac00ee70 comp rx=0 tx=0).stop 2024-04-25T01:28:00.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 --2- 172.21.15.129:0/3069502458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe4c40c8b20 0x7fe4c414b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.461+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 >> 172.21.15.129:0/3069502458 conn(0x7fe4c40acbf0 msgr2=0x7fe4c40ab480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:00.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.462+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 shutdown_connections 2024-04-25T01:28:00.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.463+0000 7fe4ca5fb640 1 -- 172.21.15.129:0/3069502458 wait complete. 2024-04-25T01:28:00.458 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T01:28:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.584+0000 7f86202b3640 1 Processor -- start 2024-04-25T01:28:00.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.584+0000 7f86202b3640 1 -- start start 2024-04-25T01:28:00.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f86202b3640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f86180c6920 0x7f86180c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:00.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f86202b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c5870 0x7f861806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:00.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f86202b3640 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86180ac7a0 con 0x7f8618157290 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f86202b3640 1 -- --> v2:172.21.15.129:3300/0 -- mon_getmap magic: 0 -- 0x7f86180c4c80 con 0x7f86180c6920 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f86202b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f86180aca40 con 0x7f86180c5870 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861d84e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f86180c6920 0x7f86180c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861d84e640 1 --2- >> v2:172.21.15.129:3300/0 conn(0x7f86180c6920 0x7f86180c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38906/0 (socket says 172.21.15.129:38906) 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861d84e640 1 -- 172.21.15.129:0/3898929992 learned_addr learned my addr 172.21.15.129:0/3898929992 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861e850640 1 --2- 172.21.15.129:0/3898929992 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c5870 0x7f861806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861d84e640 1 -- 172.21.15.129:0/3898929992 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c5870 msgr2=0x7f861806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861d84e640 1 --2- 172.21.15.129:0/3898929992 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c5870 0x7f861806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.585+0000 7f861d84e640 1 -- 172.21.15.129:0/3898929992 >> v1:172.21.15.171:6789/0 conn(0x7f8618157290 legacy=0x7f86180ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.586+0000 7f861d84e640 1 -- 172.21.15.129:0/3898929992 --> v2:172.21.15.129:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86180acd20 con 0x7f86180c6920 2024-04-25T01:28:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.586+0000 7f861d84e640 1 --2- 172.21.15.129:0/3898929992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c6920 0x7f86180c5330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8614009870 tx=0x7f861402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b52ac7cf437241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.586+0000 7f861d04d640 1 -- 172.21.15.129:0/3898929992 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614037070 con 0x7f86180c6920 2024-04-25T01:28:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.586+0000 7f861d04d640 1 -- 172.21.15.129:0/3898929992 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8614032440 con 0x7f86180c6920 2024-04-25T01:28:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.586+0000 7f861d04d640 1 -- 172.21.15.129:0/3898929992 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614032830 con 0x7f86180c6920 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 -- 172.21.15.129:0/3898929992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c6920 msgr2=0x7f86180c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 --2- 172.21.15.129:0/3898929992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c6920 0x7f86180c5330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8614009870 tx=0x7f861402a270 comp rx=0 tx=0).stop 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 -- 172.21.15.129:0/3898929992 shutdown_connections 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 --2- 172.21.15.129:0/3898929992 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c5870 0x7f861806fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 --2- 172.21.15.129:0/3898929992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c6920 0x7f86180c5330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 -- 172.21.15.129:0/3898929992 >> 172.21.15.129:0/3898929992 conn(0x7f8618066ea0 msgr2=0x7f8618067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 -- 172.21.15.129:0/3898929992 shutdown_connections 2024-04-25T01:28:00.567 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.587+0000 7f86202b3640 1 -- 172.21.15.129:0/3898929992 wait complete. 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.588+0000 7f86202b3640 1 Processor -- start 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.588+0000 7f86202b3640 1 -- start start 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 0x7f86181de750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c6920 0x7f86181df030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8618157290 0x7f86181e9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8618157670 con 0x7f86180c5870 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86180ad2e0 con 0x7f8618157290 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 -- 0x7f8618070170 con 0x7f86180c6920 2024-04-25T01:28:00.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 0x7f86181de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 0x7f86181de750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38918/0 (socket says 172.21.15.129:38918) 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 -- 172.21.15.129:0/2087183120 learned_addr learned my addr 172.21.15.129:0/2087183120 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861d84e640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c6920 0x7f86181df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e850640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8618157290 0x7f86181e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 -- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c6920 msgr2=0x7f86181df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c6920 0x7f86181df030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 -- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8618157290 msgr2=0x7f86181e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8618157290 0x7f86181e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 -- 172.21.15.129:0/2087183120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86180a9130 con 0x7f86180c5870 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f861e04f640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 0x7f86181de750 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f860800c3c0 tx=0x7f860800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080052d0 con 0x7f86180c5870 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8608005490 con 0x7f86180c5870 2024-04-25T01:28:00.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86181ec7c0 con 0x7f86180c5870 2024-04-25T01:28:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608007280 con 0x7f86180c5870 2024-04-25T01:28:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.589+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181ecd80 con 0x7f86180c5870 2024-04-25T01:28:00.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.590+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8618068fd0 con 0x7f86180c5870 2024-04-25T01:28:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.591+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8608007420 con 0x7f86180c5870 2024-04-25T01:28:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.591+0000 7f860effd640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f85fc06be30 0x7f85fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.591+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f860808a3b0 con 0x7f86180c5870 2024-04-25T01:28:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.591+0000 7f861d84e640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f85fc06be30 0x7f85fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:00.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.592+0000 7f861d84e640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f85fc06be30 0x7f85fc06e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8614034000 tx=0x7f861402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:00.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.595+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86080585f0 con 0x7f86180c5870 2024-04-25T01:28:00.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:00.776+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86180c7180 con 0x7f86180c5870 2024-04-25T01:28:01.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.461+0000 7f860effd640 1 -- 172.21.15.129:0/2087183120 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f860805c8f0 con 0x7f86180c5870 2024-04-25T01:28:01.440 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data_ec' removed 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f85fc06be30 msgr2=0x7f85fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f85fc06be30 0x7f85fc06e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8614034000 tx=0x7f861402e000 comp rx=0 tx=0).stop 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 msgr2=0x7f86181de750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 0x7f86181de750 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f860800c3c0 tx=0x7f860800c880 comp rx=0 tx=0).stop 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 shutdown_connections 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:6832/1668644098,v1:172.21.15.171:6833/1668644098] conn(0x7f85fc06be30 0x7f85fc06e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f86180c6920 0x7f86181df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.444 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8618157290 0x7f86181e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 --2- 172.21.15.129:0/2087183120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f86180c5870 0x7f86181de750 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 >> 172.21.15.129:0/2087183120 conn(0x7f8618066ea0 msgr2=0x7f86180705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:01.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.465+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 shutdown_connections 2024-04-25T01:28:01.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-25T01:28:01.467+0000 7f86202b3640 1 -- 172.21.15.129:0/2087183120 wait complete. 2024-04-25T01:28:01.469 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T01:28:01.469 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T01:28:01.469 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:01.470 INFO:tasks.ceph.mds.a.smithi129.stderr:2024-04-25T01:28:01.491+0000 7fd163e18640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i a (PID: 40008) UID: 0 2024-04-25T01:28:01.470 INFO:tasks.ceph.mds.a.smithi129.stderr:2024-04-25T01:28:01.491+0000 7fd163e18640 -1 mds.a *** got signal Terminated *** 2024-04-25T01:28:07.571 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T01:28:07.571 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T01:28:07.571 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:07.572 INFO:tasks.ceph.mds.c.smithi129.stderr:2024-04-25T01:28:07.593+0000 7f7fd0616640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i c (PID: 40012) UID: 0 2024-04-25T01:28:07.572 INFO:tasks.ceph.mds.c.smithi129.stderr:2024-04-25T01:28:07.593+0000 7f7fd0616640 -1 mds.c *** got signal Terminated *** 2024-04-25T01:28:13.672 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T01:28:13.673 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T01:28:13.673 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:13.673 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:28:13.695+0000 7fa5779e3640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i b (PID: 40568) UID: 0 2024-04-25T01:28:13.674 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-04-25T01:28:13.695+0000 7fa5779e3640 -1 mds.b *** got signal Terminated *** 2024-04-25T01:28:19.775 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T01:28:19.775 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T01:28:19.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:19.776 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:28:19.797+0000 7ff199406640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mds -f --cluster ceph -i d (PID: 40569) UID: 0 2024-04-25T01:28:19.776 INFO:tasks.ceph.mds.d.smithi171.stderr:2024-04-25T01:28:19.797+0000 7ff199406640 -1 mds.d *** got signal Terminated *** 2024-04-25T01:28:22.474 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T01:28:25.877 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T01:28:25.877 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T01:28:25.878 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T01:28:25.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:25.878 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:28:25.900+0000 7fce923c8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 0 (PID: 34565) UID: 0 2024-04-25T01:28:25.879 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:28:25.900+0000 7fce923c8640 -1 osd.0 35 *** Got signal Terminated *** 2024-04-25T01:28:25.879 INFO:tasks.ceph.osd.0.smithi129.stderr:2024-04-25T01:28:25.900+0000 7fce923c8640 -1 osd.0 35 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:28.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-25T01:28:28.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T01:28:31.979 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T01:28:31.980 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T01:28:31.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:31.980 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:28:32.002+0000 7f1f46d2c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 1 (PID: 34575) UID: 0 2024-04-25T01:28:31.981 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:28:32.002+0000 7f1f46d2c640 -1 osd.1 37 *** Got signal Terminated *** 2024-04-25T01:28:31.981 INFO:tasks.ceph.osd.1.smithi129.stderr:2024-04-25T01:28:32.002+0000 7f1f46d2c640 -1 osd.1 37 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:34.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-25T01:28:34.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-25T01:28:38.081 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T01:28:38.081 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T01:28:38.081 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:38.082 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:28:38.104+0000 7fd7e31ad640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 2 (PID: 34579) UID: 0 2024-04-25T01:28:38.082 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:28:38.104+0000 7fd7e31ad640 -1 osd.2 38 *** Got signal Terminated *** 2024-04-25T01:28:38.083 INFO:tasks.ceph.osd.2.smithi129.stderr:2024-04-25T01:28:38.104+0000 7fd7e31ad640 -1 osd.2 38 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:40.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-25T01:28:40.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-25T01:28:44.183 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T01:28:44.184 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T01:28:44.184 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:44.185 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:28:44.206+0000 7f309042f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 3 (PID: 34584) UID: 0 2024-04-25T01:28:44.185 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:28:44.206+0000 7f309042f640 -1 osd.3 39 *** Got signal Terminated *** 2024-04-25T01:28:44.185 INFO:tasks.ceph.osd.3.smithi129.stderr:2024-04-25T01:28:44.206+0000 7f309042f640 -1 osd.3 39 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:46.716 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-25T01:28:46.716 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-25T01:28:50.285 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T01:28:50.286 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T01:28:50.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:50.287 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:28:50.308+0000 7f7d4ae88640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 4 (PID: 34792) UID: 0 2024-04-25T01:28:50.287 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:28:50.308+0000 7f7d4ae88640 -1 osd.4 40 *** Got signal Terminated *** 2024-04-25T01:28:50.287 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-04-25T01:28:50.308+0000 7f7d4ae88640 -1 osd.4 40 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:52.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-25T01:28:52.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-25T01:28:56.387 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T01:28:56.388 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T01:28:56.388 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:28:56.389 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:28:56.410+0000 7fd1a868a640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 5 (PID: 34788) UID: 0 2024-04-25T01:28:56.389 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:28:56.410+0000 7fd1a868a640 -1 osd.5 41 *** Got signal Terminated *** 2024-04-25T01:28:56.389 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-04-25T01:28:56.410+0000 7fd1a868a640 -1 osd.5 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:28:58.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T01:28:58.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-25T01:29:02.489 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T01:29:02.490 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T01:29:02.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:02.491 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:29:02.512+0000 7f7efda7c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 6 (PID: 34791) UID: 0 2024-04-25T01:29:02.491 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:29:02.512+0000 7f7efda7c640 -1 osd.6 42 *** Got signal Terminated *** 2024-04-25T01:29:02.491 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-04-25T01:29:02.512+0000 7f7efda7c640 -1 osd.6 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:29:03.835 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-25T01:29:03.836 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T01:29:08.592 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T01:29:08.592 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T01:29:08.592 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:08.593 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:29:08.615+0000 7f500c6cc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 7 (PID: 34795) UID: 0 2024-04-25T01:29:08.593 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:29:08.615+0000 7f500c6cc640 -1 osd.7 42 *** Got signal Terminated *** 2024-04-25T01:29:08.593 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-04-25T01:29:08.615+0000 7f500c6cc640 -1 osd.7 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:29:09.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-25T01:29:14.693 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T01:29:14.694 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T01:29:14.694 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T01:29:14.694 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:14.728 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T01:29:14.728 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T01:29:14.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:14.729 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-04-25T01:29:14.751+0000 7fb9a2c10640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 34213) UID: 0 2024-04-25T01:29:14.759 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T01:29:14.760 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T01:29:14.760 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:29:14.760 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:14.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-25T01:29:14.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-25T01:29:14.769 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-04-25T01:29:14.791+0000 7f6e5552f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mon -f --cluster ceph -i a (PID: 34291) UID: 0 2024-04-25T01:29:14.769 INFO:tasks.ceph.mon.a.smithi129.stderr:2024-04-25T01:29:14.791+0000 7f6e5552f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T01:29:14.837 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:29:14.837 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T01:29:14.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:14.838 INFO:tasks.ceph.mon.b.smithi171.stderr:2024-04-25T01:29:14.861+0000 7f01d4d3a640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mon -f --cluster ceph -i b (PID: 34208) UID: 0 2024-04-25T01:29:14.838 INFO:tasks.ceph.mon.b.smithi171.stderr:2024-04-25T01:29:14.861+0000 7f01d4d3a640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-25T01:29:19.862 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-04-25T01:29:19.862 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-25T01:29:19.862 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-25T01:29:20.939 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T01:29:20.939 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:29:20.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:29:20.940 INFO:tasks.ceph.mon.c.smithi171.stderr:2024-04-25T01:29:20.962+0000 7fcf19a2b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mon -f --cluster ceph -i c (PID: 34210) UID: 0 2024-04-25T01:29:20.940 INFO:tasks.ceph.mon.c.smithi171.stderr:2024-04-25T01:29:20.962+0000 7fcf19a2b640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-25T01:29:21.029 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:29:21.029 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T01:29:21.030 DEBUG:teuthology.orchestra.run.smithi129:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T01:29:21.103 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:29:21.103 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T01:29:21.303 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:29:21.303 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T01:29:21.408 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:29:21.409 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T01:29:21.484 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:29:21.484 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T01:29:21.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:29:21.567 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T01:29:21.752 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:29:21.753 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T01:29:21.848 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:29:21.848 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T01:29:21.942 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:29:21.942 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T01:29:22.012 INFO:tasks.ceph:Archiving mon data... 2024-04-25T01:29:22.013 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/data/mon.a.tgz 2024-04-25T01:29:22.014 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-04-25T01:29:22.034 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.FHlMzcsgvU 2024-04-25T01:29:22.034 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FHlMzcsgvU 2024-04-25T01:29:22.167 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.FHlMzcsgvU 2024-04-25T01:29:22.247 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.FHlMzcsgvU is 330KB 2024-04-25T01:29:22.344 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.FHlMzcsgvU 2024-04-25T01:29:22.358 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/data/mon.b.tgz 2024-04-25T01:29:22.358 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-04-25T01:29:22.373 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.24DhwbvR0x 2024-04-25T01:29:22.374 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.24DhwbvR0x 2024-04-25T01:29:22.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.24DhwbvR0x 2024-04-25T01:29:22.637 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.24DhwbvR0x is 338KB 2024-04-25T01:29:22.733 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.24DhwbvR0x 2024-04-25T01:29:22.747 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/data/mon.c.tgz 2024-04-25T01:29:22.748 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-04-25T01:29:22.802 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.8GPJZcqh3P 2024-04-25T01:29:22.802 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8GPJZcqh3P 2024-04-25T01:29:22.926 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.8GPJZcqh3P 2024-04-25T01:29:23.008 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.8GPJZcqh3P is 338KB 2024-04-25T01:29:23.105 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.8GPJZcqh3P 2024-04-25T01:29:23.121 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T01:29:23.121 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T01:29:23.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T01:29:23.216 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T01:29:23.246 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T01:29:23.246 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T01:29:23.248 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/remote/smithi129/crash 2024-04-25T01:29:23.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T01:29:23.282 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/remote/smithi171/crash 2024-04-25T01:29:23.283 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T01:29:23.309 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T01:29:23.309 DEBUG:teuthology.orchestra.run.smithi129:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:23.328 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:23.398 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:23.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T01:29:23.497 INFO:tasks.ceph:Compressing logs... 2024-04-25T01:29:23.497 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:29:23.500 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43980.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44190.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43980.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47787.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47913.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47787.log.gz 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43398.log 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44064.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47913.log.gz 2024-04-25T01:29:23.583 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40017.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41856.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46949.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42102.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40711.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4%/var/log/ceph/ceph-client.admin.42102.log: -- replaced with /var/log/ceph/ceph-client.admin.46949.log.gz 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40098.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46865.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46697.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41856.log.gz 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46697.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46865.log.gz /var/log/ceph/ceph-client.admin.44274.log 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40753.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44190.log.gz 2024-04-25T01:29:23.584 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.46697.log.gz 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40499.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43980.log.gz 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43050.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46991.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46907.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46991.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28357.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46907.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46991.log.gz 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42312.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46907.log.gz 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.tmp-client.admin.28357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-04-25T01:29:23.585 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28357.log.gz 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40243.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40243.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40711.log.gz 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.2.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42024.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46571.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42024.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46739.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43405.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40140.log 2024-04-25T01:29:23.586 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46571.log.gz 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43405.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46739.log.gz 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42024.log.gz 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41588.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45556.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41630.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47619.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41588.log.gz 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47285.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41630.log: /var/log/ceph/ceph-client.admin.47619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42480.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47285.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2024-04-25T01:29:23.587 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47619.log.gz 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41630.log.gz 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42480.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47285.log.gz 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40795.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46445.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40669.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41982.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40669.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46445.log.gz 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43398.log.gz 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40627.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44106.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46823.log 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40795.log.gz 2024-04-25T01:29:23.588 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47703.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46823.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46823.log.gz 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43304.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47703.log.gz 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40669.log.gz 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44877.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42698.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.44877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43770.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40369.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log/var/log/ceph/ceph-client.admin.43770.log: 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40369.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43812.log 2024-04-25T01:29:23.589 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44022.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41982.log.gz 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40327.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45824.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40327.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40627.log.gz 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43401.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40945.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43401.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44484.log 2024-04-25T01:29:23.590 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41714.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44316.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46529.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40413.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47955.log 89.5% 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46529.log.gz 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45908.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47955.log.gz 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41672.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45908.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-04-25T01:29:23.591 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41940.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5/var/log/ceph/ceph-client.admin.47369.log: --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43048.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43058.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41355.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41814.log: /var/log/ceph/ceph-client.admin.43644.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41355.log: /var/log/ceph/ceph-client.admin.43644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39921.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42699.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46613.log 2024-04-25T01:29:23.592 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.42704.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46781.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46613.log.gz 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47411.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8%/var/log/ceph/ceph-client.admin.46781.log: -- replaced with /var/log/ceph/ceph-client.admin.44106.log.gz 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42522.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46781.log.gz 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.39840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43403.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-04-25T01:29:23.593 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47661.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43303.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47579.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43303.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47661.log.gz 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45782.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43044.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47579.log.gzgzip 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45782.log: /var/log/ceph/ceph-client.admin.43044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45866.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42703.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.45866.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46487.log 2024-04-25T01:29:23.594 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46487.log: -- replaced with /var/log/ceph/ceph-client.admin.41672.log.gz 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41714.log.gz 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41898.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46487.log.gz 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42697.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40541.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43055.log: /var/log/ceph/ceph-client.admin.44148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42694.log 2024-04-25T01:29:23.595 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34594.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.44232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34330.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/ceph-client.admin.47745.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45950.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8%/var/log/ceph/ceph-client.admin.34330.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.41940.log.gz 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.a.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43770.log.gz 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43812.log.gz 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40879.log 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44022.log.gz 2024-04-25T01:29:23.596 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.40879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41546.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.41546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34697.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42438.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.34697.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43686.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46655.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.40837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43896.log 88.7% 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.46655.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gzgzip 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43046.log 2024-04-25T01:29:23.597 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43896.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45514.log 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46655.log.gz 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45472.log 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45514.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44316.log.gz 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43854.log 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.45472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.43854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42228.log 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45682.log: -- /var/log/ceph/ceph-client.admin.47829.log 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.42228.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-04-25T01:29:23.598 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.47829.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2024-04-25T01:29:23.600 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34578.log 2024-04-25T01:29:23.600 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41023.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.1.41023.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41023.log.gz 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34407.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34618.log 2024-04-25T01:29:23.601 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34698.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34487.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34531.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34447.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34531.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-client.admin.34447.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34578.log.gz 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34618.log.gz 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-04-25T01:29:23.602 INFO:teuthology.orchestra.run.smithi171.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34698.log.gz 2024-04-25T01:29:23.605 INFO:teuthology.orchestra.run.smithi129.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz 2024-04-25T01:29:23.605 INFO:teuthology.orchestra.run.smithi129.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42102.log.gz 2024-04-25T01:29:23.605 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-04-25T01:29:23.605 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42312.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42480.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42522.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42694.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42699.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42697.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43050.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42698.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42703.log.gz 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.606 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43048.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43046.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43058.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43304.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43044.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43405.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43303.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 87.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43401.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43686.log.gz 2024-04-25T01:29:23.607 INFO:teuthology.orchestra.run.smithi129.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43403.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45514.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 87.9% 88.4% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45824.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45866.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45782.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45908.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45950.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-04-25T01:29:23.608 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41546.log.gz 2024-04-25T01:29:23.609 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-04-25T01:29:23.610 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41898.log.gz 2024-04-25T01:29:23.613 INFO:teuthology.orchestra.run.smithi129.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44484.log.gz 2024-04-25T01:29:23.622 INFO:teuthology.orchestra.run.smithi129.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34697.log.gz 2024-04-25T01:29:23.622 INFO:teuthology.orchestra.run.smithi129.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42228.log.gz 2024-04-25T01:29:23.622 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-04-25T01:29:23.623 INFO:teuthology.orchestra.run.smithi129.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 2024-04-25T01:29:23.624 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-04-25T01:29:23.624 INFO:teuthology.orchestra.run.smithi129.stderr: 88.7% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42438.log.gz 2024-04-25T01:29:23.625 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.629 INFO:teuthology.orchestra.run.smithi129.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.41355.log.gz 2024-04-25T01:29:23.633 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2024-04-25T01:29:23.641 INFO:teuthology.orchestra.run.smithi171.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34531.log.gz 2024-04-25T01:29:23.689 INFO:teuthology.orchestra.run.smithi171.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-04-25T01:29:23.689 INFO:teuthology.orchestra.run.smithi171.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34447.log.gz 2024-04-25T01:29:23.690 INFO:teuthology.orchestra.run.smithi171.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34487.log.gz 2024-04-25T01:29:23.692 INFO:teuthology.orchestra.run.smithi171.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T01:29:23.700 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40945.log.gz 2024-04-25T01:29:23.700 INFO:teuthology.orchestra.run.smithi129.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2024-04-25T01:29:23.700 INFO:teuthology.orchestra.run.smithi129.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40879.log.gz 2024-04-25T01:29:23.701 INFO:teuthology.orchestra.run.smithi129.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T01:29:23.702 INFO:teuthology.orchestra.run.smithi129.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34594.log.gz 2024-04-25T01:29:23.794 INFO:teuthology.orchestra.run.smithi129.stderr: 86.7% 88.2% 88.1% 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40017.log.gz 2024-04-25T01:29:23.794 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.794 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.794 INFO:teuthology.orchestra.run.smithi129.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39921.log.gz 89.5% 2024-04-25T01:29:23.794 INFO:teuthology.orchestra.run.smithi129.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40098.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40140.log.gz 2024-04-25T01:29:23.794 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40243.log.gz 88.8% 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40327.log.gz 89.2% 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40369.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40413.log.gz 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 88.8% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40541.log.gz 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40499.log.gz 2024-04-25T01:29:23.795 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45472.log.gz 2024-04-25T01:29:23.803 INFO:teuthology.orchestra.run.smithi129.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43854.log.gz 2024-04-25T01:29:23.804 INFO:teuthology.orchestra.run.smithi129.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 2024-04-25T01:29:23.804 INFO:teuthology.orchestra.run.smithi129.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43896.log.gz 2024-04-25T01:29:23.890 INFO:teuthology.orchestra.run.smithi129.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T01:29:23.895 INFO:teuthology.orchestra.run.smithi129.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.34330.log.gz 2024-04-25T01:29:24.074 INFO:teuthology.orchestra.run.smithi129.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T01:29:24.173 INFO:teuthology.orchestra.run.smithi171.stderr: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T01:29:24.387 INFO:teuthology.orchestra.run.smithi171.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T01:29:24.864 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-04-25T01:29:28.861 INFO:teuthology.orchestra.run.smithi129.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T01:29:29.603 INFO:teuthology.orchestra.run.smithi129.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T01:29:32.764 INFO:teuthology.orchestra.run.smithi171.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T01:29:37.815 INFO:teuthology.orchestra.run.smithi171.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T01:29:51.413 INFO:teuthology.orchestra.run.smithi171.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T01:31:05.065 INFO:teuthology.orchestra.run.smithi171.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T01:31:50.995 INFO:teuthology.orchestra.run.smithi129.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.44877.log.gz 2024-04-25T01:33:40.202 INFO:teuthology.orchestra.run.smithi129.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T01:34:25.221 INFO:teuthology.orchestra.run.smithi129.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T01:34:54.080 INFO:teuthology.orchestra.run.smithi129.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T01:34:54.081 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-25T01:34:54.082 INFO:teuthology.orchestra.run.smithi129.stderr:real 5m30.571s 2024-04-25T01:34:54.082 INFO:teuthology.orchestra.run.smithi129.stderr:user 4m6.789s 2024-04-25T01:34:54.082 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m10.111s 2024-04-25T01:41:15.463 INFO:teuthology.orchestra.run.smithi171.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T01:42:34.762 INFO:teuthology.orchestra.run.smithi171.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T01:43:53.783 INFO:teuthology.orchestra.run.smithi171.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T01:44:14.618 INFO:teuthology.orchestra.run.smithi171.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T01:44:14.620 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-04-25T01:44:14.620 INFO:teuthology.orchestra.run.smithi171.stderr:real 14m51.055s 2024-04-25T01:44:14.620 INFO:teuthology.orchestra.run.smithi171.stderr:user 5m56.292s 2024-04-25T01:44:14.620 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m16.240s 2024-04-25T01:44:14.621 INFO:tasks.ceph:Archiving logs... 2024-04-25T01:44:14.622 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/remote/smithi129/log 2024-04-25T01:44:14.623 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:45:16.306 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/remote/smithi171/log 2024-04-25T01:45:16.308 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:47:44.480 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T01:47:44.493 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T01:47:44.493 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:47:44.496 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:47:44.659 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T01:47:44.668 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T01:47:44.668 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:47:44.671 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:47:44.685 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:47:44.688 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:47:44.723 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:47:44.723 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-25T01:47:44.723 INFO:teuthology.orchestra.run.smithi171.stdout:^- hv01.front.sepia.ceph.com 2 6 377 57 +1551us[+1551us] +/- 586ms 2024-04-25T01:47:44.723 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -227us[ -220us] +/- 68ms 2024-04-25T01:47:44.723 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +267us[ +267us] +/- 75ms 2024-04-25T01:47:44.724 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:47:44.728 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:47:44.728 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-25T01:47:44.729 INFO:teuthology.orchestra.run.smithi129.stdout:^- hv01.front.sepia.ceph.com 2 6 377 54 +1547us[+1547us] +/- 586ms 2024-04-25T01:47:44.729 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -228us[ -227us] +/- 68ms 2024-04-25T01:47:44.729 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +245us[ +247us] +/- 75ms 2024-04-25T01:47:44.729 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:47:44.730 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T01:47:44.739 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T01:47:44.740 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T01:47:44.752 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:47:44.852 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:47:44.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:47:44.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:47:44.966 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:47:44.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:47:44.995 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T01:47:45.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T01:47:45.016 INFO:teuthology.task.internal:Duration was 3182.050966 seconds 2024-04-25T01:47:45.016 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T01:47:45.027 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T01:47:45.027 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:47:45.030 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:47:45.090 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:47:45.092 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:47:45.600 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T01:47:45.600 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-04-25T01:47:45.600 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:47:45.621 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T01:47:45.621 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:47:45.648 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T01:47:45.648 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:47:45.664 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:47:45.717 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T01:47:45.718 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:47:45.733 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:47:45.885 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T01:47:45.896 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T01:47:45.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:47:45.943 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:47:45.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T01:47:45.996 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:47:45.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:47:46.025 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-04-25T01:47:46.064 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-04-25T01:47:46.084 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:47:46.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:47:46.098 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:47:46.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:47:46.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T01:47:46.150 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T01:47:46.152 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/remote/smithi129 2024-04-25T01:47:46.152 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:47:46.185 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672302/remote/smithi171 2024-04-25T01:47:46.185 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:47:46.217 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T01:47:46.218 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:47:46.222 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:47:46.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T01:47:46.282 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T01:47:46.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T01:47:46.314 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T01:47:46.315 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:47:46.317 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:47:46.329 INFO:teuthology.orchestra.run.smithi129.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:47 /home/ubuntu/cephtest 2024-04-25T01:47:46.331 INFO:teuthology.orchestra.run.smithi171.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:47 /home/ubuntu/cephtest 2024-04-25T01:47:46.333 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T01:47:46.343 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T01:47:46.371 INFO:teuthology.run:Summary data: description: fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} duration: 3182.05096578598 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T01:47:46.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:47:46.444 INFO:teuthology.run:pass